Implements a formal evaluation framework for Claude Code sessions using Eval-Driven Development (EDD) principles.
The Eval Harness Skill empowers developers to treat AI-generated code with the same rigor as traditional software through Eval-Driven Development (EDD). By defining success criteria and expected behaviors before implementation, this skill provides a structured framework for running capability and regression evals, tracking reliability metrics like pass@k, and generating detailed evaluation reports. It is particularly useful for complex feature implementations where ensuring consistent AI performance and preventing regressions is critical, allowing for a systematic 'test-first' approach to AI prompting and coding.
主な機能
01Multi-modal grading including Code-based, Model-based, and Human review
02Structured project storage for eval definitions and run history
03Integrated CLI commands for defining, checking, and reporting evals
04Automated pass@k and pass^k reliability metrics to measure AI consistency
050 GitHub stars
06Standardized Capability and Regression Eval templates for structured testing
ユースケース
01Defining feature requirements as executable tests before starting AI implementation
02Measuring AI model reliability and success rates across multiple attempts (pass@k)
03Continuous regression testing to ensure new AI changes don't break existing logic
