What is Eval-Driven Development (EDD)?

EDD treats evaluations as unit tests for AI, defining expected behavior before coding to ensure high-quality, measurable results throughout the development process.

How does the pass@k metric work?

It measures the probability that at least one successful result is achieved within 'k' attempts, helping developers track the reliability of Claude's output over time.

Does this skill support manual reviews?

Yes, it includes a specific 'Human Review Required' marker for critical path items or security checks that should never be fully automated.

Where are evaluations stored?

Evaluations are stored locally in the .claude/evals/ directory, ensuring that your test definitions and history are versioned alongside your project code.

Eval Harness

Name: Eval Harness
Author: Fabio29T

byFabio29T

0•

セキュリティとテスト

Implements Eval-Driven Development (EDD) principles to systematically evaluate, track, and improve Claude's coding reliability.

Eval Harness is a formal evaluation framework designed for Claude Code sessions that treats evaluations as unit tests for AI-driven development. It enables developers to define expected behaviors before implementation begins, allowing for continuous measurement of success through pass@k metrics and rigorous regression tracking. By combining deterministic code-based checkers, model-based graders, and human review markers, it provides a structured workflow for ensuring AI-generated code meets specific capability benchmarks and maintains high standards of reliability throughout the development lifecycle.

主な機能

01Eval-Driven Development (EDD) workflow integration

02Multi-modal grading via code, AI models, and human review

03Automated capability and regression testing templates

04Reliability tracking with pass@1 and pass@k metrics

050 GitHub stars

06Standardized evaluation reporting and session logging

ユースケース

01Measuring the reliability and performance of Claude across multiple iterations

02Verifying new feature implementations meet all success criteria before merging

03Preventing regression errors in complex codebases during AI refactoring sessions

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add fabio29t/everything-claude eval-harness

For use in Claude.ai and ChatGPT

主な機能

01Eval-Driven Development (EDD) workflow integration

02Multi-modal grading via code, AI models, and human review

03Automated capability and regression testing templates

04Reliability tracking with pass@1 and pass@k metrics

050 GitHub stars

06Standardized evaluation reporting and session logging

ユースケース

01Measuring the reliability and performance of Claude across multiple iterations

02Verifying new feature implementations meet all success criteria before merging

03Preventing regression errors in complex codebases during AI refactoring sessions

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add fabio29t/everything-claude eval-harness

For use in Claude.ai and ChatGPT