Revenue Impact
Revenue impact is the dollar-lift verdict on an A/B test — the metric that catches winners that quietly tanked AOV. Here's how to compute it and why it overrides conversion rate.
Revenue Impact
The dollar lift an A/B test produces per visitor, combining conversion rate and average order value into a single revenue verdict.
Revenue impact is the change in revenue-per-visitor (RPV) between a control and a variant, expressed in currency rather than as a conversion-rate delta. It's the metric that decides whether a test actually made the store money — because a variant can win on conversion rate while quietly dragging average order value down, leaving net revenue flat or negative.
Most teams ship tests on conversion lift alone and discover the revenue damage a quarter later in the P&L. Revenue impact closes that gap by reading both levers — buyers and basket size — in the same calculation, and it's the number executives expect to see on the experiment readout.
The trap most teams fall into: a variant that pushes a discount banner above the fold lifts conversion rate from 2.4% to 2.7% — a clean 12.5% win on the dashboard. Ship it. Three weeks later, AOV has slipped from €78 to €68 because the banner trained shoppers to wait for the code, and revenue per visitor is actually down 1.2%.
Revenue impact catches this on the experiment readout, before you ship. It's the single number that reconciles conversion rate, AOV, and traffic volume into the metric the CFO actually tracks. On long checkout tests, it also surfaces the inverse case — a small CR drop offset by a much higher AOV — which a CR-only readout would kill.
Revenue Impact = (CR_variant × AOV_variant − CR_control × AOV_control) × Visitors
CR_variant
Variant conversion rate
Orders divided by visitors in the variant arm, as a decimal.
AOV_variant
Variant average order value
Mean order value in the variant arm, in currency.
CR_control
Control conversion rate
Orders divided by visitors in the control arm.
AOV_control
Control average order value
Mean order value in the control arm.
Visitors
Visitor volume
Total unique visitors exposed during the period you're projecting over (test window or annualised).
A Shopify apparel store runs a free-shipping-threshold test for 21 days, 80,000 visitors per arm.
Control CR: 2.40%
Control AOV: €78
Variant CR: 2.70%
Variant AOV: €68
Visitors per arm: 80,000
→ Per-visitor revenue: €1.872 (control) vs €1.836 (variant). Revenue impact over the test window: −€2,880, or roughly −€50k annualised at this traffic level.
Conversion rate lifted 12.5%, but AOV dropped 12.8% — the variant is a revenue loss. Do not ship.
The size of the AOV swing that flips a winning test varies by vertical. Categories with high basket flexibility — apparel, beauty multi-SKU — are most exposed, because shoppers easily drop a second item. Single-SKU categories like electronics or appliances see smaller AOV variance, so CR lift translates more cleanly into revenue lift.
How a +10% conversion rate lift translates to revenue impact across DTC verticals, by AOV change
| Vertical | AOV flat | AOV −5% | AOV −10% | AOV −15% |
|---|---|---|---|---|
| Apparel (€75 AOV) | +10.0% | +4.5% | −1.0% | −6.5% |
| Beauty (€45 AOV) | +10.0% | +4.5% | −1.0% | −6.5% |
| Home goods (€120 AOV) | +10.0% | +4.5% | −1.0% | −6.5% |
| Electronics (€280 AOV) | +10.0% | +4.5% | −1.0% | −6.5% |
| Supplements (€55 AOV, subscription) | +10.0% | +4.5% | −1.0% | −6.5% |
The math is the same across verticals — that's the point. What changes is how often each scenario shows up: apparel and beauty tests routinely move AOV by 5-15% because variants touch cross-sell modules, free-shipping thresholds, or bundle prompts. Electronics tests usually move AOV by under 2% because the basket is one item. Read your test's AOV column first, then the CR column.
Revenue impact FAQ
Because conversion rate ignores basket size. A test that converts more shoppers at smaller baskets can produce a CR win and a revenue loss simultaneously. Revenue impact is the only readout that catches this in the experiment analysis stage rather than in next quarter's P&L.
RPV is the underlying metric — revenue divided by visitors — for a single arm. Revenue impact is the delta in RPV between variant and control, multiplied out by the visitor volume you care about (test window or annualised). RPV is the rate; revenue impact is the dollar verdict.
Both. The percentage (RPV lift) tells you the effect size and is what statistical tests run on. The absolute number — euros per month or per year — is what stakeholders approve roadmaps with. A 3% RPV lift means very different things on €200k/mo and €2M/mo of test-eligible revenue.
Naively it doesn't — the formula uses gross order value. For categories with high return rates (apparel often 20-30%), compute it on net revenue or apply a category return-rate adjustment. Otherwise you'll ship variants that look like winners but generate more returns.
Yes. Revenue per visitor has higher variance than conversion rate because AOV is itself noisy. A CR test that hits 95% significance may have only 80% significance on revenue impact. Run the significance test on RPV directly when revenue is the decision metric.
Typically 1.5-3x what you'd need for conversion rate alone, because RPV variance is driven by the long tail of large orders. Plan for longer test windows on revenue-led tests, or pre-define an AOV winsorisation cap (e.g. cap orders at the 99th percentile) to tighten the variance.
Yes, and these are some of the most valuable wins. A bundle prompt or free-shipping threshold raise might convert 5% fewer shoppers but raise AOV by 15%, netting positive revenue. A CR-only readout would kill the variant; revenue impact ships it.
It doesn't directly — the formula is gross-revenue based. For tests that change product mix (e.g. pushing a higher-margin SKU), follow up with a margin-impact readout. Revenue impact is the right primary metric for most checkout, PDP, and shipping tests; margin impact matters for merchandising tests.
On a store doing €5M annually, a +1% revenue lift on a sitewide test is €50k/year — usually worth shipping if significant. On a page-specific test affecting 10% of traffic, you'd want +3-5% to clear the same bar. Set the threshold against test-eligible revenue, not total revenue.
It's the bottom-line metric in the experiment analysis readout, sitting alongside CR, AOV, and segment splits. Use CR and AOV to diagnose why the variant moved revenue (more buyers? bigger baskets?), and use revenue impact to make the ship/kill call. The three together tell the full story.
Test ideas before you ship them
Run unlimited A/B tests, attach hypotheses to outcomes, and build a searchable archive of what works — and what doesn't.