How AOV Mix Shift Explains Most RPV/CVR Disagreements
When your A/B test shows CVR up but RPV flat or down, the variant usually changed who buys — not how many. Here's how to decompose RPV = CVR × AOV and find the mix shift hiding inside.
Quick answer
When CVR moves and RPV doesn't (or moves the other way), the cause is almost always AOV mix shift: your variant pulled in more low-AOV buyers (or pushed away high-AOV ones) without changing the total purchase rate. Decompose RPV = CVR × AOV, split AOV by buyer segment, and the gap usually resolves into one obvious sub-group.
AOV mix shift
A change in the composition of buyers in an A/B test variant that moves average order value without changing conversion rate.
AOV mix shift is what happens when a test variant changes which customers complete checkout, not how many. The total conversion rate can rise, fall, or stay flat — but the basket-size distribution underneath shifts. A free-shipping threshold, a discount banner, a bundle-forward PDP, or a BOGO module can all do this: they recruit small-basket buyers, deter large-basket ones, or quietly cap units-per-order.
Because RPV = CVR × AOV, any movement in the AOV term shows up in RPV but not CVR. That mathematical asymmetry is why the two metrics disagree, and why a CVR winner can still be a contribution-margin loser.
If you've seen a variant post +4% CVR with -2% RPV and a flat or negative AOV delta, you're looking at mix shift. The lift is real in the count of orders; the value of those orders is what changed.
Why mix shift causes the disagreement
RPV decomposes cleanly into two terms: conversion rate times average order value. CVR counts orders per visitor; AOV averages basket value across those orders. They can move independently, and they often do.
A variant that lowers a perceived purchase barrier — free shipping at €40 instead of €60, a 10% banner, a single-SKU bundle — recruits buyers who weren't going to checkout. Those incremental orders are real, but they tend to sit at the low end of the basket distribution. Average down, total revenue may not move.
The arithmetic that catches most teams
If CVR rises 5% but the new orders average 30% lower basket value than your baseline buyers, AOV drops roughly proportional to the share of new buyers. RPV ends up flat or negative even though every dashboard says 'CVR winner'.
Worked example: decomposing the RPV gap
Imagine a Shopify apparel store running a free-shipping threshold test. Control offers free shipping at €60; variant drops it to €40. Both arms get 50,000 visitors over the test window.
Control: 1,500 orders (3.0% CVR) at €72 AOV → RPV €2.16. Variant: 1,650 orders (3.3% CVR) at €62 AOV → RPV €2.046. CVR is up 10% relative; RPV is down ~5%. The table below shows where the swing comes from.
RPV = CVR × AOV decomposition for a free-shipping threshold test (50,000 visitors per arm)
| Segment | Control orders | Control AOV | Variant orders | Variant AOV |
|---|---|---|---|---|
| Baskets €40–€59 (small) | 450 | €48 | 780 | €46 |
| Baskets €60–€89 (mid) | 750 | €72 | 660 | €68 |
| Baskets €90+ (large) | 300 | €105 | 210 | €98 |
| Total / weighted | 1,500 | €72.00 | 1,650 | €62.00 |
The variant added 330 small-basket buyers and lost 90 mid-basket plus 90 large-basket buyers. The CVR lift is genuine, but the new orders are worth ~€46 each while the lost orders were worth €72–€105. That's the mix shift, in numbers.
Order count by basket segment: control vs variant
Control
Variant
How to detect mix shift in your own test
Three signals, checked in order. First: compare AOV deltas. If AOV moves more than 2–3% between arms on a test that wasn't designed to change basket size, mix shift is the prior. The companion page on normal AOV variance inside a single A/B test sets the noise floor.
Second: bucket orders by basket size (or units-per-order) and compare the distributions, not just the means. A two-segment decomposition — small vs large baskets, or new vs returning buyers — usually surfaces the segment carrying the swing. Third: check units-per-order separately; BOGO and quantity-break tests hide mix shift there even when basket value looks stable.
The traffic-mix confound looks identical
Before blaming the variant, confirm the inbound traffic mix didn't shift during the test (paid spend changes, a creative refresh, a press hit). A device or channel mix change can mimic a CVR win or fake an AOV drop. Segment by source/medium and device before you decompose.
How to fix it — or ship it anyway
If RPV is your business-truth metric, mix shift means the variant lost. But shipping a CVR winner that lost RPV can still be correct when contribution margin per order is higher on the smaller baskets — that's a unit-economics call, not a test-stats call. Free-shipping threshold tests are the classic case where this decision needs to weigh shipping cost, not just revenue.
To prevent the disagreement in future tests: pre-register RPV (or contribution margin per visitor) as the primary metric before launching any test that could plausibly change basket composition. Bundle-forward PDPs, sitewide discount banners, free-shipping changes, and BOGO modules all qualify. CVR becomes a guardrail, not the decision metric.
Frequently asked questions
No. RPV is noisier than CVR because basket values have heavy-tailed variance, but mix shift is a directional bias, not noise. Mix shift moves the AOV mean systematically across arms; noise just widens the confidence interval. You can have both, and you usually do.
Roughly 2–3% AOV movement between arms is within normal test noise for most stores. Beyond ~3%, especially on tests that touched pricing, shipping, bundles, or basket-level promos, mix shift is the most likely cause of any RPV/CVR disagreement.
Yes, and bundle-forward PDPs are the textbook example. The bundle deters one-item buyers (CVR drops) but the buyers who do convert spend significantly more. Net: CVR loss, RPV win. Same mechanism, opposite sign.
On any test that could change basket composition — pricing, shipping, bundles, promos, PDP layout near upsells — yes, RPV (or contribution margin per visitor) should be primary. On tests that only affect funnel friction (form fields, page speed, copy clarity), CVR is fine because basket mix is unlikely to move.
Compute orders / visitors and revenue / orders per arm, then verify RPV = CVR × AOV holds. Then segment AOV by basket bucket, units-per-order, new-vs-returning, and device. The segment whose share changed most between arms is your mix-shift culprit.
It doesn't invalidate the CVR significance — that test is about order counts. It does mean a 'significant CVR win' isn't the same as a 'business win', and the RPV significance test (which is what you actually care about) is the one to read.
Because the banner changes who clicks through to checkout. Price-sensitive browsers convert more often; price-insensitive buyers either ignore the banner or wait for a bigger code. The buyer composition shifts even though the offer is uniform.
Yes, but it often hides in units-per-order rather than basket value. A BOGO can leave AOV roughly flat while doubling units shipped — which tanks contribution margin per order. Always check units-per-order as a separate decomposition for those tests.
Long enough for RPV — the noisier metric — to reach significance, which is typically 1.5–2x the duration a pure CVR test would need. Stopping at CVR significance on a mix-shift-prone test is the single most common way teams ship false winners.
Targeting the variant to a single segment (e.g. mobile only, returning buyers only) reduces composition swing within that segment but doesn't generalize the result. If the goal is sitewide rollout, test sitewide and pre-register RPV. Segment-targeted tests are valid but answer a narrower question.
Test ideas before you ship them
Run unlimited A/B tests, attach hypotheses to outcomes, and build a searchable archive of what works — and what doesn't.