A/B test significance calculator
Is version B really better, or did it just get lucky? Enter your visitors and conversions from any A/B test (also called a split test) to find out.
Your results so far
| Version | Visitors | Conversions | Remove |
|---|
Verdict
Enter your numbers above to see the verdict.
How many visitors do you need?
Find out roughly how many visitors each version needs before a lift of a given size would show up reliably (95% confidence, 80% chance of spotting it).
What does “statistically significant” mean?
In a split test, visitors are divided between two or more versions of a page or link, and you count how many take the action you want. Because people behave differently from one day to the next, one version can pull ahead purely by chance. Significance tells you how likely it is that the gap is real.
Reading the confidence number
If confidence is 97%, there is about a 3% chance you’d see a gap this big even if both versions were exactly equal. Most marketers wait for 95% before calling a winner.
Why the calculator sometimes says “not enough data yet”
Small numbers are noisy. Until every version has at least 300 visitors and 30 conversions, the result can swing wildly with a single sale, so this tool asks you to wait instead of guessing.
Testing more than two versions
Every extra version gives luck another chance to look like a win. With three or four versions, this calculator makes the confidence stricter (a Bonferroni adjustment), so you’re less likely to pick a false winner.
Common mistakes
- Stopping the moment you see 95%. Decide how many visitors you need first, using the estimate above, and wait for them.
- Testing too many things at once. Change one thing per test so you know what caused the difference.
- Ignoring the calendar. Run tests for at least one full week so weekday and weekend visitors are both included.
Frequently asked questions
What is a conversion?
A conversion is whatever you want the visitor to do: buy, sign up, download or click a button. Count the same action for every version.
Which statistical test does this calculator use?
A two-sided two-proportion z-test that compares each version with version A. For three or four versions, the confidence is adjusted with the Bonferroni method.
What if the result is “no clear difference”?
It means the data can’t tell the versions apart yet. Either they really perform the same, or the difference is smaller than your visitor numbers can show. You can keep the test running, or accept that the change doesn’t matter much.
How long should I run an A/B test?
Run it until each version reaches the visitor count from the estimate above, and for at least a full week. Don’t stop early just because one version is ahead.
Are my numbers saved or sent anywhere?
No. The calculation happens in your browser and nothing you type is uploaded.
Let your links pick the winner
Simple Click Tracker Pro runs split tests on your links, counts visitors and conversions for you, and can switch to the winning version automatically.
See how it works