Eval Harness
Implements an Evaluation-Driven Development (EDD) framework to measure and ensure the reliability of AI-generated code.
Eval Harness is a specialized framework for Claude Code that introduces Evaluation-Driven Development (EDD) principles to the AI coding workflow. By treating evaluations as unit tests for AI, it allows developers to define success criteria before implementation, track regressions with precision, and measure reliability using advanced metrics like pass@k. It supports code-based, model-based, and human evaluation methods, ensuring that code changes are both functional and robust across complex development sessions.
Key Features
01214,597 GitHub stars
02Automated regression tracking to prevent breaking existing functionality
03Evaluation-Driven Development (EDD) workflow integration
04Support for deterministic code-based and probabilistic model-based grading
05Reliability metrics including pass@k and pass^k success rates
06Standardized reporting for capability and regression evaluations
Use Cases
01Monitoring AI performance trends across multiple iterations of a project
02Ensuring critical security or business logic remains intact during refactoring
03Defining clear success criteria for complex features before writing any code