A/B Testing
A/B testing is a controlled experiment that compares two versions of a page or element by splitting traffic between them to learn which produces a better outcome.
In depth
An A/B test isolates a single variable, such as a headline or button copy, by sending randomized halves of your audience to a control (A) and a variant (B) at the same time. Running both simultaneously controls for seasonality and traffic-source shifts, and significance testing tells you whether the observed difference is likely real or just noise. The discipline is in changing one meaningful thing at a time so you can attribute any lift to a specific cause.
In a lead-qualification workflow, A/B testing turns opinions into evidence about what actually moves quiz starts, completions, and lead submissions. A common pitfall is stopping a test the moment it looks like a winner; ending early before reaching a predetermined sample size produces false positives that fail to replicate. Equally important is testing high-impact elements first, since rearranging trivial details rarely earns enough lift to justify the traffic an experiment consumes.
Example in practice
Frequently asked questions
How long should an A/B test run?
Run it until you reach a predetermined sample size and at least one or two full business cycles, typically one to four weeks. Ending based on time alone or stopping early when a variant looks good both undermine reliability.
How many variations can an A/B test have?
A pure A/B test compares two versions, but you can add more variants in an A/B/n test. Each extra variant splits traffic further and requires more total visitors to reach significance.
What is statistical significance in A/B testing?
It is the confidence that a measured difference between variants is real rather than random chance, often expressed as a 95% confidence level. Reaching it requires enough conversions, not just enough visitors.