What is the ab-test-setup skill for Claude Code?

It is a specialized skill that provides domain-specific guidance for planning and executing A/B tests, ensuring your experiments are mathematically sound and produce actionable results.

Does it support multivariate testing (MVT)?

Yes, the skill covers various experiment types including standard A/B tests, A/B/n tests, Multivariate tests (MVT), and Split URL tests.

Can this skill calculate my experiment's sample size?

Yes, it provides a reference framework and formulas for calculating required sample sizes based on your baseline conversion rate, minimum detectable effect (MDE), and desired statistical power.

How does this help with test analysis?

It provides a structured approach to interpreting results, checking for statistical significance (p-values), and assessing practical business impact versus implementation costs.

A/B Test Setup Expert

Name: A/B Test Setup Expert
Author: phrazzld

byphrazzld

•

Automatización de Marketing

Guides the planning, design, and implementation of statistically valid A/B tests and experiments to drive data-informed decisions.

The ab-test-setup skill transforms Claude into an expert experimentation consultant, helping you move beyond guesswork to scientific validation. It provides a comprehensive framework for creating structured hypotheses, calculating required sample sizes, and defining critical metrics like primary, secondary, and guardrail indicators. By following its rigorous methodology, you can design split tests, multivariate experiments, and feature rollouts that produce actionable insights while avoiding common pitfalls like the 'peeking problem' or insufficient statistical power.

Características Principales

01Variant design best practices for headlines, visuals, and CTAs

023 GitHub stars

03Comprehensive metric selection including primary and guardrail metrics

04Detailed test plan and results documentation templates

05Statistical sample size and test duration calculation guidance

06Structured hypothesis framework based on data and observations

Casos de Uso

01Analyzing completed test results to distinguish between statistical and practical significance

02Designing a high-impact split test for a marketing landing page to increase conversions

03Determining the traffic requirements and timeline for a new product feature experiment

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add phrazzld/claude-config ab-test-setup

For use in Claude.ai and ChatGPT

Download Skill