ICSE Empirical Experiments

•
1.1k
•

Designs and audits empirical evaluation methodologies for software engineering research papers targeting ICSE-level academic conferences.

ICSE Empirical Experiments provides specialized guidance for designing, conducting, and auditing rigorous evaluations for ICSE research-track submissions and top software engineering venues. It guides researchers through claim-evidence alignment, robust benchmark selection, fair baseline comparisons, standard non-parametric statistical testing with effect sizes (such as Vargha-Delaney Â12 and Cliff's delta), qualitative research rigor, and ablation studies for AI-for-SE systems. By enforcing threats-driven design and sanity checks before experiments are run, this skill ensures that empirical research papers withstand scrutiny from demanding software engineering peer reviewers.

主な機能

01Evaluates claim-evidence proportionality to eliminate ungrounded general claims
02Applies standard SE statistical methods including non-parametric tests and effect sizes
031,079 GitHub stars
04Provides ablation and data contamination audit checklists for AI and LLM tools
05Establishes fair baseline protocols with matched compute budgets and execution parameters
06Enforces threats-driven pre-hoc study design to eliminate validity risks before paper writing

ユースケース

01Planning ablation studies, cost reporting, and contamination safeguards for AI-for-SE research
02Designing empirical benchmarks, statistical tests, and baseline setups for a new software engineering tool
03Auditing the experimental evaluation section of an ICSE research-track manuscript before submission