Minimum Cohort Size To Trust A Channel-Level Retention Curve
The 200-customer rule for trusting a channel-level retention curve — plus what to do when Pinterest or TikTok never gets there.
Quick answer
For a 180-day retention point on a single channel-month cohort to be trustworthy, you want ~200 acquired customers. Below 100 the 95% confidence band is wider than most channel-to-channel differences you're trying to detect; below 50 the curve is essentially noise. If a channel never clears the threshold, pool 2-3 consecutive months before reallocating spend.
Minimum cohort size to trust a channel-level retention curve
The smallest number of customers acquired in a single channel-month before that channel's retention curve is reliable enough to act on.
Channel-level retention curves plot the share of a monthly acquisition cohort still purchasing at 30, 60, 90 and 180 days, split by the channel that acquired them. The minimum cohort size is the point at which the sampling error around each retention point shrinks enough that real differences between channels can be separated from random variation.
For most €1-15M stores the working threshold is around 200 customers per channel-month for a 180-day read, dropping to ~100 for shorter horizons. Below that, confidence intervals on the retention rate become wider than the effect sizes you're trying to detect — which is exactly the situation that produces confident, wrong spend reallocations.
The pain shows up in a specific way. You open a channel-level retention chart in GA4 or a BI tool, see Pinterest's 180-day curve sitting 8 points above Meta's, and start drafting a budget shift. Then someone asks how many Pinterest customers that line is based on. The answer is 43.
Why small channel cohorts lie
Retention at any horizon is a proportion. The sampling error on a proportion scales with 1/√n, so halving your cohort widens the confidence band by ~40%. At n=40 with an observed 25% retention, the 95% CI runs roughly 13% to 40% — a 27-point spread.
That's wider than almost any real channel difference you care about. Meta vs Pinterest 180-day retention in the same vertical typically differs by 3-8 points. You cannot detect a 5-point true difference when each channel's estimate is drifting ±13 points on noise alone.
The 40-customer failure mode
A 40-customer channel cohort will happily show you a 180-day retention rate anywhere from 15% to 45% purely by chance — see our deep-dive on why a 40-customer cohort lies about 180-day retention. Reallocating €10K/month on that signal is a coin flip with extra steps.
How to detect whether your curves are trustworthy
First check: for every channel-month you're comparing, count the cohort size and write it next to the retention number. Any curve built on fewer than 100 customers gets a visible warning flag; anything under 50 gets pulled from the chart entirely.
Second check: overlay the 95% confidence interval as a shaded band around each curve. When bands from two channels overlap at the 180-day point, the difference is not decidable at that sample size. The confidence interval math for a retention rate at a given cohort size is a two-line normal approximation you can compute in the same query.
Cohort size vs. 95% CI half-width around an observed 25% 180-day retention
| Cohort size (n) | Observed retention | 95% CI half-width | Verdict |
|---|---|---|---|
| 40 | 25% | ±13.4 pp | Noise — do not act |
| 100 | 25% | ±8.5 pp | Directional only |
| 200 | 25% | ±6.0 pp | Usable for reallocation |
| 500 | 25% | ±3.8 pp | Strong signal |
| 1,000 | 25% | ±2.7 pp | Best-in-class |
CI half-width shrinks with √n — but slowly past 200
How to fix a below-threshold channel
The default fix is pooling: combine 2-3 consecutive channel-months into a single cohort. A Pinterest channel doing 70 acquired customers per month becomes a rolling 210-customer cohort at the quarter mark, which clears the 200 threshold cleanly. See our note on pooling channel-months without losing signal for when this breaks.
Pooling has one hazard: seasonality. Q4-acquired customers on a beauty store retain differently from Q1 customers on the same channel, so pooling across the Nov-Feb boundary blends two different populations. If you must pool across a season, split the report at the seasonal break and treat them as distinct.
The vertical also matters. A repeat-consumables store (coffee, supplements, pet food) sees enough repeat purchases inside 60 days that a smaller cohort — often ~120 — is enough for a stable 180-day read. A considered-purchase high-AOV store (furniture, premium electronics) needs closer to 300, because most customers make zero repeats and the signal is thin.
The decision rule we recommend
Set two thresholds in your BI layer: n≥200 = show the curve, n=100-199 = show but flag as directional, n<100 = suppress from the reallocation view entirely. When a channel is chronically below 100, escalate to the 'what to do when a channel never hits the minimum' decision — usually either pool quarterly, treat it as a brand channel outside the retention lens, or accept you can't compare it.
Experiments to run before you reallocate
Run a hindcast: take last quarter's actual retention curves, compute what your reallocation rule would have done, then check how the newly-favored channel actually performed the following quarter. If the channel's 180-day retention swung by more than the CI half-width you computed, your threshold is too low.
Also run a bootstrap check: resample each channel-month cohort 1,000 times with replacement and plot the distribution of 180-day retention. Channels whose bootstrapped distribution overlaps a competitor's by more than 25% are not distinguishable — you're picking between them on noise, and the right move is to hold spend flat and grow the smaller channel's volume until it clears the threshold.
Frequently asked questions
Rule of thumb. It's calibrated for a 180-day retention point around 20-30% (the typical DTC range), where the CI half-width at n=200 is roughly 6pp — narrow enough to detect channel-to-channel differences in the 6-10pp range you'd actually act on. If your observed retention is very low (say 8%) the CI is narrower at the same n, and you can trust smaller cohorts.
You can drop the threshold to ~100 customers per channel-month. Shorter horizons have higher retention rates and more observations per customer, so the sampling error is smaller. The 200 number is specifically tuned for the 180-day point, which is where most reallocation decisions get made.
An A/B test compares two variants under randomised assignment; you're comparing two channels under self-selected assignment. That means even with enough sample size, channel differences reflect audience selection (who Pinterest reaches vs Meta) as much as channel quality. Sample size gets you a trustworthy estimate; it does not get you causal attribution.
No — pooling across channels destroys the exact signal you're trying to measure. Pool within one channel across months (or across similar geos, if you're running Shopify Markets). If a channel is chronically under-scale, the honest move is to either grow it or exclude it from retention-based reallocation.
Roughly 700-900 customers per channel-month, depending on baseline retention. A 3pp difference is small relative to typical CI half-widths at n=200 (~6pp each), so you need substantially more volume. Our page on the cohort-size threshold for detecting a 3pp retention lift walks through the math.
Show the retention curve as a solid line only when n≥200, dashed when 100-199, and hide it entirely below 100. Always print the cohort size on the same axis label. A shaded 95% CI band around each curve makes non-technical stakeholders self-correct when they see overlapping bands.
Same principle, larger threshold. LTV is even noisier than retention because a single high-AOV repeat customer can move the mean significantly. For channel-level LTV comparisons at 180 days, plan on 300-500 per channel-month before the estimates are stable, and use median LTV alongside mean to spot skew.
Treat Q4-acquired customers as a distinct population from the rest of the year. They tend to look artificially loyal in the first 90 days (holiday re-purchase) and then drop hard. Pool Q4 months together, and compare Q4 to Q4 year-over-year rather than Q4 to Q1 — this is the seasonality-contaminated retention curve problem.
You can shrink the effective threshold if you have a strong prior — e.g. six quarters of historical data on that channel giving you a stable baseline. In practice most €1-15M stores don't have clean enough historical data to justify the prior, so the frequentist rule of thumb (n≥200) is safer than a Bayesian model with a shaky prior.
Quarterly. Cohort sizes drift as spend shifts, and a channel that cleared 200/month in Q1 can slide to 130/month by Q3 without anyone noticing. Add a monthly monitor that flags any channel whose trailing-3-month cohort size dropped below 200, and re-decide whether to pool, exclude, or grow that channel.
Track CAC, channels, and funnel conversion in one place
Metricuno connects ad spend, funnel events, and revenue so you can see CAC by channel, cohort, and campaign — without stitching together five tools.