ASE Experiment Designer
Designs and audits empirical evaluations for Automated Software Engineering (ASE) papers to meet rigorous peer-review standards.
ASE Experiment Designer assists software engineering researchers in planning, executing, and auditing empirical evaluations for top-tier venues like IEEE/ACM ASE. It provides structured guidance on selecting real-world subject systems, establishing fair runnable baselines under equal compute budgets, isolating learned components through ablation studies, validating correctness oracles against overfitting, mitigating LLM data contamination, and reporting statistically sound effect sizes.
主要功能
01Guides selection and provenance tracking for real-world software subjects
02Aligns evaluation metrics directly with specific software automation claims
03Enforces LLM contamination checks, oracle verification, and statistical dispersion reporting
04Structures ablation studies to isolate the marginal value of learned components
051,069 GitHub stars
06Establishes fair, budget-matched baseline comparisons with runnable tools
使用场景
01Structuring revision responses to address reviewer concerns regarding baseline fairness or subject validity
02Designing empirical evaluation frameworks for LLM-based software engineering tools
03Auditing an ASE research paper evaluation section prior to conference submission