A/B Test Sample Size
1215
Calculate required sample size for A/B tests with statistical power analysis
NOTEModel Overview
🧪 A/B TEST SIZING
Determines required sample size:
• Baseline conversion rate
• Minimum detectable effect
• Desired statistical power
Uses standard power formula.
VARIABLEBaseline Conversion
beta
Current conversion rate (~5%)
baseline
VARIABLERelative Lift
uniform
Minimum detectable effect (10-20%)
lift
CONSTANTStatistical Power
0.8
80% power (standard)
power
CONSTANTSignificance Level
0.05
5% significance (standard)
alpha
FORMULATreatment Rate
baseline * (1 + lift)
Expected treatment conversion
baseline
lift
treatment
FORMULAPooled Variance
baseline * (1 - baseline) + treatment * (1 - treatment)
Combined variance
baseline
treatment
variance
FORMULASample Size
2 * variance * pow(1.96 / alpha + 0.84 * power, 2) / pow(treatment - baseline, 2)
Users per variant (using power/alpha)
variance
alpha
power
treatment
baseline
sample_size
OUTPUTRequired Sample
Users per Variant
About the creator
The team behind Carlo. We believe everyone deserves tools to reason about risk and uncertainty.
What is Carlo?
Carlo is a visual tool for Monte Carlo simulation. Model uncertainty by dragging probability distributions, connecting them visually, and running thousands of scenarios instantly.