What is the ab-test-setup skill?

It is a specialized capability for Claude Code that provides frameworks and best practices for designing, implementing, and analyzing statistically valid A/B tests.

What types of tests does this skill support?

The skill supports standard A/B tests, A/B/n (multiple variants), Multivariate Testing (MVT), and Split URL testing.

How does it help with result analysis?

It provides a rigorous checklist for interpreting results, checking for statistical significance (p-values), and evaluating secondary and guardrail metrics before making a decision.

Can it help calculate sample sizes for tests?

Yes, it includes reference tables and guidance for determining the required sample size per variant based on baseline conversion rates and desired lift.

A/B Test Setup & Experimentation

Name: A/B Test Setup & Experimentation
Author: Strat-Ai-Gize-Marketing-Automation

byStrat-Ai-Gize-Marketing-Automation

0•

营销自动化

Designs and implements statistically valid A/B tests and experiments to optimize conversion rates and product performance.

This skill provides an expert framework for running data-driven experiments, helping users move beyond intuition-based changes to structured, scientific testing. It guides you through the entire experimentation lifecycle, including building strong hypotheses, calculating required sample sizes, defining primary and guardrail metrics, and choosing between client-side or server-side implementation. By enforcing statistical rigor and preventing common pitfalls like 'peeking,' it ensures that your product decisions are backed by reliable, actionable data.

主要功能

01Comprehensive metric selection for primary, secondary, and guardrails

02Statistical sample size and test duration calculation guidance

030 GitHub stars

04Structured hypothesis framework (Observation, Change, Outcome, Metrics)

05Technical implementation strategies for client-side and server-side tests

06Result analysis checklists for statistical significance and confidence intervals

使用场景

01Planning a landing page split test to improve signup conversion rates

02Analyzing experiment data to decide if a variant should be fully deployed

03Determining the minimum traffic required to detect a 10% lift in clicks

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add strat-ai-gize-marketing-automation/marketing-skills ab-test-setup

For use in Claude.ai and ChatGPT

Download Skill