A/B Testing
A/B testing splits your traffic between two (or more) versions of a page or asset to determine which converts better, with statistical confidence.
What it is
You serve version A to half of users and version B to the other half (with random assignment). Conversion rates are tracked per variant. After enough sample size, you can declare a winner with a confidence interval — typically 95% confidence is the bar.
Why it matters
A/B testing replaces opinion with evidence. "My designer says blue buttons convert" is a hypothesis; an A/B test is the evidence. Without it, you're making product and marketing decisions on gut feel — and most gut-feel changes are neutral or negative.
Common A/B test types
- Headline copy
- Hero image / video
- CTA button copy + color
- Form length / field order
- Pricing display (e.g., $4,000/mo vs starting at $4,000)
- Social proof placement
- Long-form vs short-form pages
Frequently asked questions
How much traffic do I need?
Roughly 100 conversions per variant for a 20% lift detection. Lower lifts need more traffic. For low-traffic pages, focus on bigger swings or test on higher-volume entry points first.
Can I test 5+ variants at once?
Yes (multivariate), but you need a lot more traffic. Multivariate is best when you have a strong base of traffic and want to test interaction effects.
What if my A/B test shows no winner?
That's a useful result. Either the change didn't matter or you're underpowered. Pick a bolder hypothesis next.
