How are performance baselines calculated?

The skill executes a test multiple times and calculates the median value to establish a stable, statistically significant baseline for comparison.

Which programming languages does Metric Runner support?

It supports a wide range of environments including Node.js, Python, npm, pnpm, bun, and test frameworks like Jest, Vitest, and Pytest.

What is the Ralph Loop integration?

Metric Runner acts as the verification engine for Ralph Loop, providing the feedback needed for the autonomous agent to decide if a code change met its optimization goals.

How does the skill ensure security during command execution?

It uses a safe execution script that validates command formats and rejects shell metacharacters like semicolons, pipes, or ampersands to prevent command injection.

Can I use this for non-numeric tests?

Yes, it supports boolean success/failure formats as well as JSON objects containing text-based reasons and error messages for standard functional testing.

Metric Runner

Name: Metric Runner
Author: ponderingBGI

byponderingBGI

•

Security & Testing

Executes verification tests and extracts quantitative metrics to drive autonomous code optimization loops.

About

The Metric Runner skill provides a robust framework for Claude Code to execute test suites, parse performance data, and track progress during autonomous optimization cycles. Specifically designed for the Ralph Loop workflow, it ensures secure command execution, calculates baseline measurements using multi-run median values, and supports multiple runtime environments including Node.js, Python, and various modern test runners. This skill is essential for developers utilizing AI agents to iteratively improve code performance, latency, or reliability based on objective, measurable results rather than qualitative guesses.

Key Features

Automatic parsing of numeric and boolean metrics from standardized JSON outputs.
Comprehensive progress tracking with YAML-based metric history logging.
1 GitHub stars
Secure test execution with shell metacharacter protection and command validation.
Built-in comparison logic for achieving specific performance goals and targets.
Baseline establishment using multi-run median calculations for performance stability.

Use Cases

Establishing objective performance baselines before initiating iterative, AI-driven code improvements.
Automating performance tuning by measuring latency or throughput changes during AI refactoring.
Verifying autonomous code changes against pass/fail criteria in a git worktree isolation loop.

About

Key Features

Automatic parsing of numeric and boolean metrics from standardized JSON outputs.
Comprehensive progress tracking with YAML-based metric history logging.
1 GitHub stars
Secure test execution with shell metacharacter protection and command validation.
Built-in comparison logic for achieving specific performance goals and targets.
Baseline establishment using multi-run median calculations for performance stability.

Use Cases

Establishing objective performance baselines before initiating iterative, AI-driven code improvements.
Automating performance tuning by measuring latency or throughput changes during AI refactoring.
Verifying autonomous code changes against pass/fail criteria in a git worktree isolation loop.