What are the stopping rules for?

Stopping rules allow you to safely end an experiment early if a result is already significant or clearly futile, using methods like O'Brien-Fleming boundaries to keep your statistical error rates in check.

Can I use this for A/B testing sample size calculation?

Yes, it includes comprehensive power analysis tools using statsmodels to calculate the required sample size for t-tests, ANOVA, and proportion-based A/B tests before you begin data collection.

What is pre-registration in this skill?

Pre-registration is a pattern that helps you document your hypothesis and analysis plan before looking at the data, which prevents common biases like HARKing (Hypothesizing After Results are Known).

How does this skill ensure reproducibility?

The skill provides standardized code patterns to fix random seeds across Python and NumPy and automatically records the versions of libraries like Pandas and Scikit-Learn to ensure consistent environments.

Scientific Experiment Design

Name: Scientific Experiment Design
Author: Yeachan-Heo

byYeachan-Heo

•

120

•

数据科学与机器学习

Implements best practices for designing, documenting, and analyzing reproducible scientific experiments within data science workflows.

Scientific Experiment Design is a specialized skill for Claude Code that provides a structured framework for conducting statistically sound research. It automates essential reproducibility steps like random seed management and environment logging while guiding users through advanced statistical methodologies. From power analysis for sample size determination to pre-registration patterns that prevent p-hacking, this skill ensures that experiments—ranging from A/B tests to machine learning model evaluations—are rigorous, valid, and easily documented for peer review or stakeholder reporting.

主要功能

01Alpha-spending functions and O'Brien-Fleming boundaries for interim analysis

02Reproducibility templates for random seeds and environment recording

03Sequential Probability Ratio Test (SPRT) for continuous experiment monitoring

04A priori and post-hoc power analysis for statistical significance

05Pre-registration frameworks to distinguish confirmatory vs. exploratory findings

06120 GitHub stars

使用场景

01Implementing rigorous stopping rules for clinical or user behavior experiments

02Designing statistically valid A/B tests with pre-calculated sample sizes

03Setting up reproducible machine learning training pipelines with fixed environment states

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add yeachan-heo/my-jogyo experiment-design

For use in Claude.ai and ChatGPT

Download Skill