Can this skill calculate sample sizes for my tests?

Yes, it includes quick-reference tables and guidance for determining the required sample size based on your baseline conversion rate and desired lift.

Does it support both client-side and server-side testing?

Yes, the skill covers implementation strategies for both client-side JavaScript modifications and more robust server-side experimentation.

What does the A/B Test Setup skill help with?

It provides a comprehensive framework for planning, implementing, and analyzing A/B and multivariate tests to ensure they produce statistically valid and actionable data.

What is the 'Peeking Problem' mentioned in the skill?

The peeking problem refers to stopping a test early after looking at preliminary results, which often leads to false positives; this skill provides guidance on how to avoid this error.

How do I define a strong hypothesis using this skill?

The skill uses a structured template: 'Because [observation], we believe [change] will cause [outcome] for [audience], measured by [metrics].'

A/B Test Setup & Experimentation

Name: A/B Test Setup & Experimentation
Author: lev-os

bylev-os

0•

マーケティングオートメーション

Designs and implements statistically valid A/B tests and experiments to drive data-informed product improvements.

The A/B Test Setup skill transforms Claude into an experimentation specialist, helping users move beyond guesswork to rigorous data-driven decision-making. It provides a structured framework for defining clear hypotheses, determining necessary sample sizes, selecting primary and guardrail metrics, and designing effective variants. Whether you are optimizing landing page copy or testing complex server-side features, this skill ensures your experiments follow statistical best practices to produce actionable, reliable results while avoiding common pitfalls like the 'peeking problem'.

主な機能

01Structured hypothesis framework based on data-driven observations

02Sample size and duration calculation guidance for statistical power

03Metric selection strategy including primary, secondary, and guardrail metrics

04Comprehensive pre-launch checklists and implementation best practices

05Post-test analysis templates for interpreting statistical significance

060 GitHub stars

ユースケース

01Planning a new product experiment with a statistically valid sample size

02Designing variant copy or UI elements for a landing page split test

03Analyzing experiment results to determine business impact and next steps

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add lev-os/agents ab-test-setup

For use in Claude.ai and ChatGPT

Download Skill