Experiment Analysis
Designs and analyzes rigorous A/B tests using frequentist frameworks to make clear, data-driven ship or kill product decisions.
Experiment Analysis is an end-to-end framework for product managers and data teams to plan, power, and interpret online controlled experiments without p-hacking or premature peeking. Grounded in industry best practices from Kohavi, Tang, and Xu, this skill guides you through formulating falsifiable hypotheses, selecting single primary metrics alongside guardrails, calculating required sample sizes and runtimes, and checking for sample ratio mismatch (SRM). It delivers an honest five-way read evaluating statistical significance against minimum detectable effects (MDE), outputting structured experiment markdown reports that turn ambiguous experiment dashboards into confident ship, kill, or iterate decisions.