Winning Variant
A winning variant is the version of an A/B test that outperforms the others on the primary metric with enough statistical confidence to act on.
In depth
Declaring a winner is more than spotting the highest number on a dashboard. A genuine winner has reached the pre-defined sample size and significance threshold, and its lead is stable rather than a momentary spike caused by random variance or a novelty effect. Calling winners early, the most common mistake in CRO, repeatedly promotes variants that regress to the mean once full traffic arrives.
A winning variant should also be evaluated against the right metric, not a flattering proxy. A quiz funnel variant that boosts quiz starts but lowers qualified lead submissions has not actually won, even though its top-of-funnel number looks impressive. Once a true winner is confirmed, it becomes the new control and baseline, and the improvement is locked in before the next round of testing begins.
Example in practice
Frequently asked questions
How do I know a variant truly won?
A true winner has reached your pre-set sample size and significance level, and its lead stays stable rather than fluctuating day to day. If the gap shrinks as traffic grows, it was likely noise rather than a real effect.
Can a variant win on the wrong metric?
Yes, a variant can lift a top-of-funnel number while harming the metric that actually matters, like qualified leads or revenue. Always judge winners on the primary business metric, not a flattering proxy.
What do I do after declaring a winner?
Roll the winner out to all traffic and make it your new control, then monitor briefly to confirm the lift holds. The next experiment starts from this improved baseline so gains compound over time.