Journal of Software Experiment Designer
Designs, refines, and audits empirical software engineering experiments and evaluations aligned with Journal of Software (《软件学报》) peer-review standards.
Tailored specifically for submissions to the Journal of Software (《软件学报》), this skill guides researchers and developers through designing and presenting rigorous empirical software experiments. It enforces research question (RQ) alignment, authentic dataset selection, competitive baseline comparisons, statistical significance and effect size calculations, ablation studies, LLM data contamination defenses, repository mining provenance, and four-dimensional threats-to-validity analyses across various software subdisciplines.
Key Features
01Guides statistical validation using hypothesis testing and effect size measures such as Cliff's delta and Cohen's d
02Provides domain-customized evaluation checklists for SE, system software, databases, PL, and software security
03Mitigates LLM benchmark data contamination and establishes rigorous component ablation experiments
041,086 GitHub stars
05Aligns Research Questions (RQs) directly with empirical evaluation metrics and core paper contributions
06Structures comprehensive four-category threats to validity covering construct, internal, external, and conclusion validity
Use Cases
01Auditing and polishing empirical evaluation sections prior to submitting to the Journal of Software (《软件学报》)
02Formulating clear research questions (RQs) and documenting threats to validity in academic software publications
03Designing robust baseline comparisons and statistical validation for empirical software engineering studies