Connect

ICSME Experiment Auditor

brycewang-stanfordbybrycewang-stanford
•
1.1k
•
Learning & Documentation

Audits and designs rigorous empirical evaluations for IEEE ICSME papers with robust repository mining provenance, fair baselines, and statistical validity.

ICSME Experiments helps software engineering researchers design, refine, and audit empirical evaluations specifically tailored for IEEE International Conference on Software Maintenance and Evolution (ICSME) submissions. Because ICSME operates without a revision round, this skill ensures empirical evidence strictly aligns with maintenance claims before submission. It guides researchers through mining software repository (MSR) provenance tracking, contamination-aware LLM evaluation ablations, survivorship and history confound controls, fair baseline benchmarking, and SE-standard statistical reporting.

Key Features

01Enforces baseline fairness with equal compute budgets and tuned competing techniques
02Aligns maintenance claims with proportional empirical evidence and real system change histories
03Structures contamination-aware LLM evaluation protocols, prompt caching, and ablation checks
041,079 GitHub stars
05Verifies mining software repository (MSR) provenance, repository SHAs, and filtering hygiene
06Generates comprehensive readiness audits covering statistical effect sizes and confound mitigation

Use Cases

01Designing contamination-free benchmark experiments for LLM-based code maintenance tools
02Validating baseline fairness, dataset provenance, and statistical effect sizes in evolution studies
03Pre-submission auditing of empirical evaluation sections for ICSME and MSR research papers