Does this skill require manual cleanup after running?

No, the skill includes a mandatory Phase 5 cleanup process that automatically closes PRs/issues and removes git worktrees used during the benchmark.

What specific metrics does the benchmark track?

It tracks efficiency (duration, turns, cost, tool calls), quality (1-5 code scores via reviewer agents), and workflow adherence (sub-agent calls and step compliance).

What models can this skill evaluate?

The skill is optimized for comparing Claude models like Opus and Sonnet within the Benchmark Suite V3 framework, but it can be configured to test various models supported by the underlying runner.

Where are the benchmarking results stored?

Results are stored locally in the runtime benchmark directory and summarized in a markdown report, which can also be automatically uploaded as a GitHub issue or release artifact.

Model Evaluation Benchmark

Name: Model Evaluation Benchmark
Author: rysweet

byrysweet

•

데이터 과학 및 ML

Orchestrates comprehensive AI model evaluation benchmarks to measure efficiency, code quality, and workflow adherence.

The Model Evaluation Benchmark skill provides an automated framework for reproducing the Benchmark Suite V3 reference implementation. It enables developers to perform side-by-side comparisons of different AI models, such as Claude 3.5 Sonnet and Opus, by measuring critical performance indicators like token cost, execution duration, and tool usage. This skill automates the entire benchmarking lifecycle—from task execution and reviewer-agent analysis to report generation and environment cleanup—ensuring that evaluations of agentic workflows remain reproducible, data-driven, and resource-efficient.

주요 기능

01Agentic code quality scoring via reviewer sub-agents

02Mandatory cleanup protocols for PRs, issues, and git worktrees

03Automated execution of Benchmark Suite V3 tasks

04Automated GitHub report generation and artifact archiving

0516 GitHub stars

06Side-by-side model comparison for efficiency and cost

사용 사례

01Generating reproducible benchmarking reports for engineering stakeholders

02Comparing performance and cost between different Claude model versions

03Validating agentic workflow adherence and tool call efficiency

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add rysweet/amplihack model-evaluation-benchmark

For use in Claude.ai and ChatGPT

Download Skill