Can I use this for programmatic labeling?

Yes, the skill includes implementation patterns for Snorkel-based weak supervision, allowing you to use labeling functions and heuristics to generate training data rapidly.

Does it support active learning?

It implements a two-phase active learning loop that combines uncertainty sampling with core-set diversity selection to maximize the efficiency of your labeling budget.

What is a Claude Code Skill?

A Claude Code Skill is a specialized capability that provides domain-specific guidance and implementation patterns, enhancing Claude's performance for specific engineering tasks.

How does it detect errors in existing datasets?

The skill uses Cleanlab-inspired patterns to perform 'confident learning,' which identifies likely mislabeled examples by comparing labels against out-of-sample predicted probabilities.

How does this skill help with label quality?

It provides standardized patterns for Inter-Annotator Agreement (IAA) monitoring and gold label injection to detect annotator fatigue and guideline drift in real-time.

Data Labeling Patterns

Name: Data Labeling Patterns
Author: HermeticOrmus

byHermeticOrmus

0•

数据科学与机器学习

Implements expert annotation pipeline designs, quality assurance protocols, and scalable labeling systems for machine learning workflows.

This skill equips Claude with specialized patterns for architecting high-quality data labeling operations. It provides robust frameworks for designing annotation guidelines, implementing active learning sampling strategies (uncertainty and diversity), and managing weak supervision pipelines via Snorkel. By incorporating rigorous quality metrics like Inter-Annotator Agreement (IAA) monitoring, gold label injection for annotator QA, and automated error detection via Cleanlab, this skill ensures that ML engineers can produce the high-fidelity training data required for production-grade AI models.

主要功能

01Confident learning patterns for identifying and correcting label errors

02Weak supervision pipelines for programmatic labeling at scale

03Structured annotation guideline templates with edge-case decision trees

040 GitHub stars

05Automated Inter-Annotator Agreement (IAA) reporting and monitoring

06Multi-phase active learning loops using uncertainty and diversity sampling

使用场景

01Designing and scaling manual annotation workflows for complex NLP or CV tasks

02Cleaning existing datasets by identifying mislabeled examples using predictive probabilities

03Implementing hybrid labeling strategies that combine heuristics with human-in-the-loop validation

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add hermeticormus/libremlops-claude-code labeling-patterns

For use in Claude.ai and ChatGPT

主要功能

01Confident learning patterns for identifying and correcting label errors

02Weak supervision pipelines for programmatic labeling at scale

03Structured annotation guideline templates with edge-case decision trees

040 GitHub stars

05Automated Inter-Annotator Agreement (IAA) reporting and monitoring

06Multi-phase active learning loops using uncertainty and diversity sampling

使用场景

01Designing and scaling manual annotation workflows for complex NLP or CV tasks

02Cleaning existing datasets by identifying mislabeled examples using predictive probabilities

03Implementing hybrid labeling strategies that combine heuristics with human-in-the-loop validation

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add hermeticormus/libremlops-claude-code labeling-patterns

For use in Claude.ai and ChatGPT