Case study
Water bottle brand
A homepage A/B test: trust badges and social proof against the current version. The variant won with 50% more conversions and 58% more revenue per visitor.
Context
A water bottle brand selling on Shopify that was already improving conversion with experiments, not hunches. The homepage got most of the traffic, so any improvement there multiplied across the whole funnel.
Problem
The team suspected that adding trust badges and social proof to the homepage would help it convert, but that was a hypothesis, not a fact. Redesigning the page blind could raise sales or lower them, and without measuring there was no way to know which.
Constraint
Test on real traffic without hurting the experience or slowing sales, and decide on statistical significance, not opinion.
What I built
I set the test up as a template test in Intelligems: the current homepage as the control against a version with trust badges and social proof as the variant. Each one got half of the real traffic, without touching checkout or the rest of the site.
The badge design wasn’t the point. The point was to stop deciding blind. Instead of rebuilding the page on a hunch and crossing our fingers, I let the numbers decide: conversion rate as the primary metric, with revenue and profit per visitor as supporting metrics.
It ran for about two weeks, with roughly 9,300 visitors split between the two versions. The social proof variant won consistently and with high significance: a 99% probability of beating the control on revenue per visitor, and practically certain on conversion.
The recommendation was simple and easy to defend: send the variant to 100% of traffic. Not because it looked better, but because it was proven to sell more.
Result
The variant won: 50% more conversions (from 1.80% to 2.69%) and 58% more revenue per visitor, with a 99% probability of beating the current version.