Connect

ICSE Reproducibility & Artifact Verification

brycewang-stanfordbybrycewang-stanford
•
1.1k
•
Learning & Documentation

Guides the preparation of verifiable, open-science compliant replication packages and data availability sections for ICSE research papers.

ICSE Reproducibility helps software engineering researchers meet ICSE's strict open-science and verifiability criteria during the review process. It provides structured guidance on drafting compliant Data Availability statements, preparing anonymized replication packages, and establishing determinism ledgers for tool, LLM, and repository-mining studies. By running pre-submission verifiability audits and identifying provenance decay risks, this skill ensures research claims are independently auditable, maximizes the Verifiability and Transparency review score, and streamlines post-acceptance artifact evaluation.

Key Features

01Generates compliant, honest Data Availability statements for full or justified partial sharing
02Provides study-specific reproducibility checklists for tool evaluations, repo mining, and LLM techniques
03Establishes determinism ledgers tracking seeds, dependency lockfiles, hardware, and container digests
04Audits double-anonymous replication packages to prevent identity leaks and broken dependencies
05Executes a five-question pre-submission self-audit to preempt negative reviewer comments on verifiability
061,079 GitHub stars

Use Cases

01Drafting transparent Data Availability sections that properly justify proprietary or IRB-restricted data
02Preparing an anonymized, runnable replication package for an ICSE research-track submission
03Pinning provenance and caching raw outputs for LLM-based software engineering benchmarks to prevent decay