Does this skill help with environment management?

It verifies the existence of requirements.txt, environment.yml, or lock files and documents hardware details to ensure the compute environment can be replicated.

What does the reproducibility score represent?

The score (out of 17) evaluates your experiment across environment, data, code, and result benchmarks, ranking it from 'Poor' to 'Excellent' with specific instructions for improvement.

How does the skill verify random seeds?

The skill scans your experiment code for common stochastic calls such as np.random.seed, random_state parameters, and library-specific seeds (e.g., torch.manual_seed) to ensure all sources of randomness are controlled.

Can it detect non-deterministic GPU operations?

Yes, the skill flags known non-deterministic operations, such as multi-threaded data loading or GPU training without deterministic flags like torch.use_deterministic_algorithms().

Reproducibility Checklist

Name: Reproducibility Checklist
Author: andikarachman

byandikarachman

•

Data Science & ML

Ensures machine learning experiments are fully reproducible by verifying random seeds, data hashes, and environment configurations.

The Reproducibility Checklist is a specialized tool for data scientists and ML engineers to validate that their experiments can be accurately recreated by others. It provides a structured framework for auditing random seeds, library versions, data integrity via SHA-256 hashes, and Git code states. By scoring experiments across six critical categories, it helps teams catch non-deterministic behavior and environment drift before code is moved to production, ensuring scientific rigor and reliable model deployment.

Key Features

01Comprehensive verification of global and stochastic random seeds

02Library version auditing and Python environment capture

03Git state and commit history tracking for code versioning

04Data integrity validation using SHA-256 file hashing

05Automated reproducibility scoring with actionable recommendations

069 GitHub stars

Use Cases

01Reviewing an ML experiment before merging it into a shared repository

02Standardizing experiment documentation for collaborative research

03Troubleshooting performance discrepancies between local and cloud environments

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add andikarachman/data-science-plugin reproducibility-checklist

For use in Claude.ai and ChatGPT

Key Features

01Comprehensive verification of global and stochastic random seeds

02Library version auditing and Python environment capture

03Git state and commit history tracking for code versioning

04Data integrity validation using SHA-256 file hashing

05Automated reproducibility scoring with actionable recommendations

069 GitHub stars

Use Cases

01Reviewing an ML experiment before merging it into a shared repository

02Standardizing experiment documentation for collaborative research

03Troubleshooting performance discrepancies between local and cloud environments

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add andikarachman/data-science-plugin reproducibility-checklist

For use in Claude.ai and ChatGPT