In what language are the reports generated?

While the internal reasoning is performed in English, all user-facing outputs, status reports, and summaries are generated in Japanese.

How are the evaluation criteria stored?

All definitions and execution logs are stored locally in the .claude/evals/ directory within your project repository for easy versioning.

What is the primary benefit of the eval-driven skill?

It provides a structured way to ensure that AI-generated code meets specific functional requirements and doesn't break existing features by using measurable evaluation criteria.

What metrics does this skill track?

It specifically tracks capability metrics like pass@3 (success within three attempts) and regression metrics like pass^3 (consistent success across three runs).

Eval-Driven Development Manager

Name: Eval-Driven Development Manager
Author: YuriNakayama

byYuriNakayama

0•

Security & Testing

Manages the eval-driven development workflow by defining, executing, and reporting on feature-specific success criteria.

The Eval-Driven Development Manager skill enables developers to implement a rigorous evaluation-driven workflow directly within Claude Code. It automates the creation of evaluation templates, tracks capability and regression pass rates, and generates detailed progress reports. Designed for high-stakes feature development, it ensures that new capabilities meet specific success criteria (like pass@k metrics) while maintaining system stability through regression checks, providing all user-facing output and summaries in Japanese.

Key Features

01Automated log management and cleanup for optimized workflows

02Standardized Markdown templates for defining capability and regression evals

03Automated execution and logging of evaluation checks

04Comprehensive Japanese-language reporting with pass/fail metrics

050 GitHub stars

06Centralized dashboard to list and track status across multiple features

Use Cases

01Defining and verifying success criteria for complex new feature implementations

02Running automated regression checks to ensure existing functionality remains intact

03Generating professional Japanese status reports for project stakeholders

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add yurinakayama/research eval-driven

For use in Claude.ai and ChatGPT