Which tools is this skill compatible with?

It provides implementation guidance for popular testing and analytics stacks including PostHog, Optimizely, VWO, LaunchDarkly, and Split.io.

Can this skill help calculate sample sizes?

Yes, it provides quick-reference tables and guidance for determining the required traffic per variant based on your baseline conversion rate and the minimum detectable effect (MDE) you wish to see.

How does it handle the 'peeking problem'?

The skill educates users on the risks of stopping tests early and provides strict guidelines on pre-committing to sample sizes to avoid false positives and statistical errors.

What testing types are supported?

The skill supports standard A/B tests, A/B/n tests for multiple variants, Multivariate Testing (MVT) for combinations, and Split URL tests for entirely different page versions.

What is the hypothesis framework used by this skill?

It uses a structured format: 'Because [observation/data], we believe [change] will cause [expected outcome] for [audience], measured by [metrics].' This ensures every test has a clear, logical foundation.

A/B Test Setup & Strategy

Name: A/B Test Setup & Strategy
Author: Facundorios108

byFacundorios108

0•

Automatización de Marketing

Designs and implements statistically valid A/B tests and experiments to optimize conversion rates and user behavior.

This skill transforms Claude into an expert experimentation consultant, providing a rigorous framework for planning, executing, and analyzing A/B tests. It guides users through creating strong hypotheses, calculating required sample sizes to ensure statistical significance, and selecting primary, secondary, and guardrail metrics. Whether you are testing landing page copy or complex server-side feature flags, this skill ensures your experiments produce actionable data while avoiding common pitfalls like the 'peeking problem' or testing too many variables at once.

Características Principales

01Sample size and duration calculators to ensure statistical rigor

02Implementation patterns for both client-side and server-side testing

03Structured hypothesis generation framework based on data and observations

04Result analysis templates for interpreting statistical significance and confidence intervals

05Metric selection guidance for primary, secondary, and guardrail KPIs

060 GitHub stars

Casos de Uso

01Reducing churn by experimenting with different pricing structures and value propositions

02Optimizing landing page conversion rates through headline and CTA variations

03Validating new feature impact using controlled rollouts and split testing

Características Principales

01Sample size and duration calculators to ensure statistical rigor

02Implementation patterns for both client-side and server-side testing

03Structured hypothesis generation framework based on data and observations

04Result analysis templates for interpreting statistical significance and confidence intervals

05Metric selection guidance for primary, secondary, and guardrail KPIs

060 GitHub stars

Casos de Uso

01Reducing churn by experimenting with different pricing structures and value propositions

02Optimizing landing page conversion rates through headline and CTA variations

03Validating new feature impact using controlled rollouts and split testing