ATC Systems Experiments Auditor

•
1.1k
•

Guides the experimental design and evaluation auditing for ACM SIGOPS ATC systems research papers with rigorous benchmarking standards.

The ATC Systems Experiments Auditor skill equips systems researchers and developers with rigorous guidelines for designing and reviewing experimental evaluations for the ACM SIGOPS Annual Technical Conference (USENIX ATC). It helps users systematically map empirical evidence to system claims, establish fair and tuned baselines, configure realistic testbeds, and balance end-to-end results with isolating microbenchmarks. By enforcing tail-latency (p99/p999) tracking, variance reporting, transparent cost accounting, and strict provenance tracking, this skill ensures your systems evaluation withstands intense peer review.

Key Features

01Establishes fair baseline comparisons at matched operational and resource costs
02Standardizes provenance tracking for hardware, software versions, and workload seeds
03Maps architectural and performance claims directly to expected empirical evidence
04Enforces rigorous tail latency (p99/p999), variance, and negative-result reporting
05Balances end-to-end macro workloads with mechanism-isolating microbenchmarks
061,074 GitHub stars

Use Cases

01Structuring reproducible benchmarking suites and performance metric logging for computer systems research
02Designing the evaluation section of a systems paper targeted at ACM SIGOPS / USENIX ATC
03Auditing an experimental evaluation against common systems reviewer objections before submission