How are results recorded for later analysis?

The skill logs metrics into a JSONL format, placing the primary metric at the top level and secondary metrics in a nested object for easy querying.

What output format does the Metric Extraction skill expect?

The skill parses lines that follow the format 'METRIC name=value', where the name can include alphanumeric characters, dots, or underscores.

How does the skill handle different measurement units?

It automatically infers units from metric name suffixes; for example, '_ms' is treated as milliseconds and '_kb' as kilobytes.

What is the difference between primary and secondary metrics?

The primary metric is the optimization target used to decide if a run is kept or discarded, while secondary metrics are tracked for monitoring tradeoffs.

What happens if a run fails to produce a primary metric?

If the primary metric is missing from the output, the skill treats the run as a crash, indicating the benchmark did not produce the expected data.

Metric Extraction for Autoresearch

Name: Metric Extraction for Autoresearch
Author: pbdeuchler

bypbdeuchler

•

분석 및 모니터링

Parses and tracks performance metrics from experiment outputs to drive automated optimization loops and benchmarks.

The Metric Extraction skill enables Claude to programmatically interpret structured output from the autoresearch.sh utility. It automatically identifies primary optimization targets and secondary monitoring metrics, infers measurement units—such as milliseconds, microseconds, or kilobytes—and logs findings into a standardized JSONL format. This skill is essential for developers running iterative benchmarks or automated experiments where consistent data tracking and tradeoff analysis are required to make informed coding decisions based on empirical evidence.

주요 기능

0111 GitHub stars

02Parses structured METRIC name=value output lines automatically

03Infers measurement units from name suffixes like _ms, _µs, and _kb

04Tracks and registers new secondary metrics across multiple experimental runs

05Distinguishes between primary optimization targets and secondary metrics

06Standardizes experiment logging into queryable JSONL formats

사용 사례

01Monitoring resource usage tradeoffs during iterative code optimization

02Automated performance benchmarking and regression detection

03Collecting domain-specific signals to guide AI-driven research tasks

주요 기능

0111 GitHub stars

02Parses structured METRIC name=value output lines automatically

03Infers measurement units from name suffixes like _ms, _µs, and _kb

04Tracks and registers new secondary metrics across multiple experimental runs

05Distinguishes between primary optimization targets and secondary metrics

06Standardizes experiment logging into queryable JSONL formats

사용 사례

01Monitoring resource usage tradeoffs during iterative code optimization

02Automated performance benchmarking and regression detection

03Collecting domain-specific signals to guide AI-driven research tasks