What is Eval-Driven Development (EDD)?

EDD is a workflow where you define success criteria and evaluation tests for an AI agent before starting the actual implementation, similar to Test-Driven Development (TDD) for traditional software.

Where are the evaluation results and definitions stored?

All eval definitions, run logs, and baselines are stored locally within your project in the .claude/evals/ directory, allowing them to be versioned alongside your code.

How does pass@k measurement work in this skill?

It calculates the probability that the AI succeeds at least once within 'k' attempts, providing a more realistic measure of agent reliability than a single pass/fail result.

Can I use deterministic tests with Eval Harness?

Yes, the skill supports Code-Based Graders that use shell commands, grep, and existing test runners to provide deterministic verification of Claude's outputs.

Eval Harness

Name: Eval Harness
Author: saar120

bysaar120

0•

安全与测试

Implements an evaluation-driven development (EDD) framework to measure and verify Claude's performance across tasks.

The Eval Harness skill brings Eval-Driven Development (EDD) to Claude Code, allowing developers to treat AI evaluations as unit tests. It enables the definition of clear pass/fail criteria before implementation, tracks reliability through pass@k metrics, and prevents regressions with automated suites. By supporting code-based, model-based, and human graders, it provides a robust infrastructure for benchmarking agent performance and ensuring high-quality, stable outputs during complex AI-assisted coding sessions.

主要功能

01Automated regression test suite generation

02Deterministic, Model-based, and Human grading systems

030 GitHub stars

04Reliability tracking with pass@k and pass^k metrics

05Standardized eval reporting and history logging

06Eval-Driven Development (EDD) workflow implementation

使用场景

01Benchmarking Claude's performance across different model versions

02Protecting critical code paths with automated regression testing

03Defining success criteria for complex feature implementations before coding

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add saar120/skillgate-registry eval-harness

For use in Claude.ai and ChatGPT

主要功能

01Automated regression test suite generation

02Deterministic, Model-based, and Human grading systems

030 GitHub stars

04Reliability tracking with pass@k and pass^k metrics

05Standardized eval reporting and history logging

06Eval-Driven Development (EDD) workflow implementation

使用场景

01Benchmarking Claude's performance across different model versions

02Protecting critical code paths with automated regression testing

03Defining success criteria for complex feature implementations before coding

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add saar120/skillgate-registry eval-harness

For use in Claude.ai and ChatGPT