This calculator uses the Two-Proportion Z-Test to determine if the difference between conversion rates in your A/B test is statistically significant.

The Null Hypothesis

In A/B testing, we start with the null hypothesis (H0): There is no difference between the conversion rates of the control (A) and variant (B). The alternative hypothesis (H1): There IS a difference.

Statistical Significance

We calculate a p-value representing the probability of observing the difference if the null hypothesis were true. Common confidence levels:

• 90% confidence (α = 0.10): Less strict, useful for exploratory tests
• 95% confidence (α = 0.05): Industry standard for most tests
• 99% confidence (α = 0.01): Very strict, used for high-stakes decisions

The Peeking Problem

One of the most critical issues in A/B testing is 'peeking' - checking results before reaching the predetermined sample size. This increases the false positive rate dramatically. Always determine your sample size before starting a test.

Sample Size Matters

A test with 100 visitors per variation is unlikely to detect small differences. For detecting a 5% relative uplift with 95% confidence, you typically need 10,000+ visitors per variation.