What is Eval-Driven Development (EDD)?

EDD treats evals as the unit tests for AI development, where you define expected behavior and success criteria before prompting the AI to implement the feature.

How does the Eval Harness measure reliability?

It uses pass@k metrics (at least one success in k attempts) and pass^k metrics (consecutive successes) to provide a statistical measure of AI reliability.

Can I use this for regression testing?

Yes, it specifically includes a regression eval type designed to ensure that new changes do not break existing functionality compared to a defined baseline.

Where are evaluation files stored?

Evals are stored as first-class artifacts within your project directory under .claude/evals/, including definitions, logs, and baselines.

What types of graders can I use?

The skill supports code-based graders for deterministic checks, model-based graders for qualitative evaluation, and human graders for manual review.

Eval Harness for Claude Code

Name: Eval Harness for Claude Code
Author: XD3an

byXD3an

0•

보안 및 테스팅

Implements an Eval-Driven Development framework to test, measure, and validate AI-generated code performance and reliability.

The Eval Harness skill introduces Eval-Driven Development (EDD) to the Claude Code environment, treating evaluations as the essential unit tests for AI development. It allows developers to define rigorous success criteria before implementation, run continuous capability and regression tests, and track reliability using pass@k metrics. By standardizing grading through deterministic code checks, model-based reasoning, or human review, this skill ensures that AI-generated solutions meet specific quality bars and remain stable over time.

주요 기능

01Deterministic code-based and model-based grading systems

020 GitHub stars

03Standardized evaluation reporting and baseline management

04Automated regression testing to prevent performance degradation

05Eval-Driven Development (EDD) workflow integration

06Reliability tracking using pass@k and pass^k metrics

사용 사례

01Running automated tests to ensure AI updates don't break existing functionality

02Defining detailed success criteria for complex features before AI implementation

03Quantifying the reliability and consistency of AI outputs across multiple attempts

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add xd3an/cc-plus eval-harness

For use in Claude.ai and ChatGPT

Download Skill