How to set one up
Start with a written hypothesis: “if the headline states the result instead of the service, more people will fill in the form”. Choose the metric that decides, usually conversion rate. Split traffic in half at random and let it run.
One variable per test. If you change headline, image and button together, even when you win you will not know why.
When a result can be trusted
A difference can appear by chance, especially with little data. Statistical significance tells you how unlikely a gap this size would be if both versions actually performed the same. The usual threshold is 95%, which means a p-value below 0.05.
Everything is calculated in your browser: nothing is saved or sent.
The calculator uses a two-proportion z-test, the standard method for comparing two conversion rates.
Mistakes that skew the test
Stopping as soon as one version pulls ahead: if you check the data every day, sooner or later a “significant” difference shows up by chance. Decide in advance how many visitors you need.
Running for less than a full week. Monday and Saturday bring different audiences, and a three-day test mixes them up.
Testing details on low traffic. With few visits a month, test the big things: the offer, the headline, the page structure. Button colour can wait. More in our guide to landing page anatomy.
Frequently asked questions
How long should an A/B test run?
At least one or two full weeks, and in any case until it reaches the number of visitors decided at the start. Duration depends on traffic and on the size of difference you want to detect.
What is the difference between A/B and multivariate testing?
An A/B test compares two versions that differ in one thing. A multivariate test tries several elements and their combinations at once, but needs much more traffic.
Can I run an A/B test on Meta campaigns?
Yes. Meta has a dedicated tool that splits the audience between versions, so the same people do not see both ads.
Related terms
Want to know which numbers matter in your business?
Seven short questions. The founders reply within 48 hours.
Apply