Check whether the difference between your control and variant conversion rates is statistically significant.
Frequently Asked Questions
How do you calculate A/B test statistical significance?
This uses a two-proportion z-test: it compares the conversion rates of your control and variant, accounting for sample size, to calculate a p-value and confidence level for the observed difference.
What does "95% confidence" mean?
It means there's only a 5% probability the observed difference happened by random chance alone. A result below the 95% threshold (p < 0.05) is the most commonly used bar for declaring a winner.
Why is my test not significant even though the variant did better?
Small sample sizes produce a lot of random noise. A real difference needs enough visitors and conversions before you can be statistically confident it isn't just chance.