How does this skill prevent 'peeking' at test results?

It enforces pre-defined sample sizes and durations upfront, explicitly warning against stopping tests early or changing success criteria once the experiment has launched.

Can I use this for multivariate testing (MVT)?

Yes, the skill helps you select the appropriate test type, including A/B, A/B/n, Multivariate, and Split URL tests based on your available traffic volume and complexity requirements.

What is a 'Hard Gate' in this Claude Code skill?

A Hard Gate is a mandatory pause in the workflow where the user must confirm specific criteria, such as a finalized hypothesis or metric definition, before Claude allows the process to move to the implementation phase.

What are guardrail metrics in A/B testing?

Guardrail metrics are critical indicators that must not degrade during a test. They act as a safety net to ensure that improving one metric (like clicks) doesn't accidentally hurt another (like page load speed or churn).

A/B Test Setup Guide

Name: A/B Test Setup Guide
Author: claudiodearaujo

byclaudiodearaujo

0•

分析と監視

Enforces a rigorous, data-driven framework for designing and validating A/B tests through mandatory statistical gates and hypothesis locking.

The A/B Test Setup skill transforms Claude into a disciplined experiment designer, ensuring that every split test is statistically sound and strategically aligned before any code is written. By guiding users through mandatory 'Hard Gates'—including hypothesis locking, metric definitions, and sample size calculations—this skill prevents common experimental errors such as 'peeking,' inadequate power, and ill-defined success criteria. It is an essential tool for developers and growth engineers who need to move beyond 'guessing' and implement high-integrity experiments that yield actionable, trustworthy business insights.

主な機能

01Standardized documentation templates for recording outcomes and institutional learnings.

020 GitHub stars

03Comprehensive metric hierarchy setup including Primary, Secondary, and Guardrail metrics.

04Mandatory Hypothesis Lock to prevent goalpost-shifting and ensure experimental integrity.

05Execution Readiness Gates that block implementation until all statistical requirements are met.

06Automated Sample Size and Minimum Detectable Effect (MDE) calculation prompts.

ユースケース

01Implementing safety-first UI changes by defining guardrail metrics to prevent regressions.

02Standardizing experimental documentation across a development team to avoid repeating failed tests.

03Designing conversion rate optimization (CRO) experiments with high statistical confidence.

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add claudiodearaujo/sistema-de-narra-o-de-livro-front ab-test-setup

For use in Claude.ai and ChatGPT

Download Skill