Does it support multivariate testing (MVT)?

Yes, it provides guidance for A/B, A/B/n, and multivariate tests, including the traffic requirements and complexity considerations for each.

How does this skill help with A/B testing?

It provides a comprehensive framework for experimentation, including hypothesis generation, statistical power calculation, and metric selection to ensure tests are valid.

What are guardrail metrics?

Guardrail metrics are secondary data points (like bounce rate or support tickets) monitored to ensure that a variant doesn't cause unintended harm while trying to improve the primary metric.

Can it calculate how long an A/B test should run?

Yes, it uses baseline conversion rates, daily traffic, and the minimum detectable effect (MDE) to estimate required sample sizes and test duration.

What is the 'peeking problem' in A/B testing?

Peeking refers to stopping a test as soon as you see significance before reaching the pre-calculated sample size, which often leads to false positives and inflated results.

A/B Test Setup & Experimentation

Name: A/B Test Setup & Experimentation
Author: coreyhaines31

bycoreyhaines31

•

4,091

•

마케팅 자동화

Designs and implements statistically valid A/B tests, split tests, and experimentation frameworks for marketing and product growth.

The ab-test-setup skill equips Claude with the methodologies of a growth engineer and CRO expert. It provides a structured environment for planning experiments, from generating data-backed hypotheses and calculating required sample sizes to defining primary and guardrail metrics. Whether you are running simple copy tests or complex server-side multivariate experiments, this skill ensures your testing process is rigorous, avoids common pitfalls like early peeking, and produces actionable business intelligence.

주요 기능

01Sample size and test duration estimation calculators

02Post-test analysis templates and learning repository management

03Primary, secondary, and guardrail metric selection guidance

044,091 GitHub stars

05Client-side and server-side implementation strategies

06Structured hypothesis framework (Observation/Change/Effect)

사용 사례

01Designing variant copy and visual hierarchy for high-traffic pricing pages

02Calculating the statistical power needed for a new product feature rollout

03Planning a landing page split test to improve conversion rates

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add coreyhaines31/marketingskills ab-test-setup

For use in Claude.ai and ChatGPT

Download Skill