Why would the skill refuse to proceed with a test?

The skill will refuse to proceed if the traffic is insufficient to detect the MDE, if the baseline rate is unknown, or if multiple variables are changed without a proper multivariate design.

Does this skill help with sample size calculation?

Yes, it requires you to define your baseline rate, MDE, and significance levels to estimate the required sample size and expected duration before the test starts.

What is a 'Hard Gate' in this skill?

A Hard Gate is a mandatory checkpoint where the user must explicitly confirm specific test parameters, such as the hypothesis or primary metric, before the skill allows the workflow to proceed to the next stage.

Can I use this for multivariate testing?

While it defaults to standard A/B testing, the skill supports A/B/n, Multivariate (MVT), and Split URL tests if your traffic volume supports the required complexity.

A/B Test Setup Guide

Name: A/B Test Setup Guide
Author: claudiodearaujo

byclaudiodearaujo

Analytics & Monitoring

Enforces statistical rigor and structured workflows for designing, executing, and analyzing valid A/B tests.

About

The A/B Test Setup skill for Claude Code transforms experimentation from guesswork into a disciplined engineering process. It implements mandatory 'hard gates' for hypothesis validation, metric definition, and sample size estimation to ensure every test is statistically sound before implementation begins. By guiding users through pre-requisites, validity checks, and guardrail metrics, this skill prevents common pitfalls like 'peeking' at results or running underpowered tests. It is an essential tool for product teams and growth engineers who prioritize data-driven decision-making and long-term learning over vanity metrics.

Key Features

0 GitHub stars
Comprehensive metric framework including primary, secondary, and guardrail metrics.
Statistical power and sample size estimation based on Minimum Detectable Effect (MDE).
Strict execution gates that prevent implementation until design is finalized.
Mandatory Hypothesis Lock to prevent moving goalposts during testing.
Automated result interpretation logic based on statistical significance and guardrail health.

Use Cases

Creating a standardized documentation repository for team-wide experimentation learning.
Designing valid experiments for UI/UX changes to ensure conversion uplift.
Setting up guardrail metrics to prevent feature releases from harming core business KPIs.

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add claudiodearaujo/sistema-de-narra-o-de-livro-front ab-test-setup

For use in Claude.ai and ChatGPT

Download Skill

GitHub