What types of graders are supported?

The harness supports deterministic code-based graders (like bash scripts or test runners), model-based graders (using Claude to evaluate logic), and human review flags for manual verification.

What is Eval-Driven Development (EDD)?

EDD is a methodology where you define expected AI behavior through formal evaluations before beginning implementation, similar to how Test-Driven Development (TDD) works for traditional software.

How does the pass@k metric work in this skill?

The pass@k metric calculates the success rate of at least one correct solution being generated within 'k' attempts, helping you measure the reliability of the AI's output.

Can I use this for existing projects?

Yes, you can define regression evals for your existing codebase to ensure that Claude's updates or refactors do not introduce bugs into previously working features.

Eval-Driven Development (EDD) Harness

Name: Eval-Driven Development (EDD) Harness
Author: arabicapp

byarabicapp

•

보안 및 테스팅

Implements a formal evaluation framework for Claude Code sessions to ensure code reliability through capability and regression testing.

The eval-harness skill introduces Eval-Driven Development (EDD) to Claude Code, treating evaluations as the 'unit tests' of AI development. It enables developers to define rigorous success criteria before implementation, track performance using pass@k metrics, and perform automated regression checks to ensure new changes don't break existing functionality. By utilizing a mix of code-based, model-based, and human graders, this skill provides a structured workflow for building more reliable, production-ready AI agents and complex software features.

주요 기능

01Structured EDD workflow (Define, Implement, Evaluate, Report)

021 GitHub stars

03Multi-modal grading including code-based and model-based checks

04Persistent evaluation storage and historical run logging

05Capability and regression evaluation templates

06Reliability metrics tracking with pass@k and pass^k scoring

사용 사례

01Measuring the reliability of AI-generated solutions across multiple attempts to improve consistency

02Automating regression testing to ensure AI modifications don't break legacy functionality

03Defining success criteria before starting complex feature implementations to guide Claude's logic

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add arabicapp/everything-claude-code eval-harness

For use in Claude.ai and ChatGPT

Download Skill