What are the stopping rules for?

Stopping rules allow you to safely end an experiment early if a result is already significant or clearly futile, using methods like O'Brien-Fleming boundaries to keep your statistical error rates in check.

Can I use this for A/B testing sample size calculation?

Yes, it includes comprehensive power analysis tools using statsmodels to calculate the required sample size for t-tests, ANOVA, and proportion-based A/B tests before you begin data collection.

What is pre-registration in this skill?

Pre-registration is a pattern that helps you document your hypothesis and analysis plan before looking at the data, which prevents common biases like HARKing (Hypothesizing After Results are Known).

How does this skill ensure reproducibility?

The skill provides standardized code patterns to fix random seeds across Python and NumPy and automatically records the versions of libraries like Pandas and Scikit-Learn to ensure consistent environments.

Scientific Experiment Design

Name: Scientific Experiment Design
Author: Yeachan-Heo

byYeachan-Heo

•

120

数据科学与机器学习

Implements best practices for designing, documenting, and analyzing reproducible scientific experiments within data science workflows.

关于

Scientific Experiment Design is a specialized skill for Claude Code that provides a structured framework for conducting statistically sound research. It automates essential reproducibility steps like random seed management and environment logging while guiding users through advanced statistical methodologies. From power analysis for sample size determination to pre-registration patterns that prevent p-hacking, this skill ensures that experiments—ranging from A/B tests to machine learning model evaluations—are rigorous, valid, and easily documented for peer review or stakeholder reporting.

主要功能

Alpha-spending functions and O'Brien-Fleming boundaries for interim analysis
Reproducibility templates for random seeds and environment recording
Sequential Probability Ratio Test (SPRT) for continuous experiment monitoring
A priori and post-hoc power analysis for statistical significance
Pre-registration frameworks to distinguish confirmatory vs. exploratory findings
120 GitHub stars

使用场景

Implementing rigorous stopping rules for clinical or user behavior experiments
Designing statistically valid A/B tests with pre-calculated sample sizes
Setting up reproducible machine learning training pipelines with fixed environment states

关于

主要功能

Alpha-spending functions and O'Brien-Fleming boundaries for interim analysis
Reproducibility templates for random seeds and environment recording
Sequential Probability Ratio Test (SPRT) for continuous experiment monitoring
A priori and post-hoc power analysis for statistical significance
Pre-registration frameworks to distinguish confirmatory vs. exploratory findings
120 GitHub stars

使用场景

Implementing rigorous stopping rules for clinical or user behavior experiments
Designing statistically valid A/B tests with pre-calculated sample sizes
Setting up reproducible machine learning training pipelines with fixed environment states