A/B Test Setup Guide

概要

The A/B Test Setup skill for Claude Code transforms experimentation from guesswork into a disciplined engineering process. It implements mandatory 'hard gates' for hypothesis validation, metric definition, and sample size estimation to ensure every test is statistically sound before implementation begins. By guiding users through pre-requisites, validity checks, and guardrail metrics, this skill prevents common pitfalls like 'peeking' at results or running underpowered tests. It is an essential tool for product teams and growth engineers who prioritize data-driven decision-making and long-term learning over vanity metrics.

主な機能

• 0 GitHub stars
• Comprehensive metric framework including primary, secondary, and guardrail metrics.
• Statistical power and sample size estimation based on Minimum Detectable Effect (MDE).
• Strict execution gates that prevent implementation until design is finalized.
• Mandatory Hypothesis Lock to prevent moving goalposts during testing.
• Automated result interpretation logic based on statistical significance and guardrail health.

ユースケース

• Creating a standardized documentation repository for team-wide experimentation learning.
• Designing valid experiments for UI/UX changes to ensure conversion uplift.
• Setting up guardrail metrics to prevent feature releases from harming core business KPIs.