Designs and implements statistically valid A/B tests and experiments to optimize conversion rates and user behavior.
This skill transforms Claude into an expert experimentation consultant, providing a rigorous framework for planning, executing, and analyzing A/B tests. It guides users through creating strong hypotheses, calculating required sample sizes to ensure statistical significance, and selecting primary, secondary, and guardrail metrics. Whether you are testing landing page copy or complex server-side feature flags, this skill ensures your experiments produce actionable data while avoiding common pitfalls like the 'peeking problem' or testing too many variables at once.
Key Features
01Sample size and duration calculators to ensure statistical rigor
02Implementation patterns for both client-side and server-side testing
03Structured hypothesis generation framework based on data and observations
04Result analysis templates for interpreting statistical significance and confidence intervals
05Metric selection guidance for primary, secondary, and guardrail KPIs
060 GitHub stars
Use Cases
01Reducing churn by experimenting with different pricing structures and value propositions
02Optimizing landing page conversion rates through headline and CTA variations
03Validating new feature impact using controlled rollouts and split testing