Pre-Mortem Analysis Checklist
Pre-mortem analysis flips post-mortem thinking forward: assume the test or launch has already failed, then list the reasons why. Use it before locking your next sprint's experiment roadmap.
Pre-Mortem Analysis
A risk-surfacing exercise where the team assumes a test or launch has failed and works backwards to identify why — before committing budget.
Pre-mortem analysis is a structured pre-launch exercise borrowed from decision science. The team is asked to imagine, vividly, that the experiment or feature launch has already failed six weeks from now. Everyone independently writes down the reasons it failed, then the list is consolidated, scored, and used to either kill the idea, redesign it, or add monitoring.
It sits upstream of experiment prioritization in most CRO workflows: you run it on shortlisted hypotheses before assigning sprint capacity. The goal is to convert hindsight-style insight (which usually arrives too late, in the post-mortem) into foresight you can act on while the test is still cheap to change.
The mechanic is simple but the psychology matters. When you ask a team "what could go wrong?", optimism bias and politeness compress the answer. When you ask "it's eight weeks from now and the test bombed — write down why", the same people produce a far richer list. Research by Deborah Mitchell and Gary Klein found prospective hindsight increased the number of correctly identified failure causes by around 30%.
For an e-commerce experimentation team, the payoff is concrete. A typical Shopify A/B test on a product detail page eats 3-6 weeks of traffic before reaching significance. A 20-minute pre-mortem that catches one tracking gap, one mobile regression, or one segment cannibalisation issue pays for itself the first time you run it.
It's not a brainstorm — it's a kill gate
The pre-mortem only works if the team is genuinely empowered to stop the test based on what surfaces. If the outcome is "we noted the risks and shipped anyway", you've added a meeting, not a safeguard. Decide upfront which failure causes are blockers, which are monitor-and-go, and who owns the call.
The 6-step pre-mortem template
Step 1 — Frame the failure scenario. Write a single sentence: "It is [date 6-8 weeks from now]. We shipped the [variant name] test on [surface]. It lost decisively, or — worse — looked flat while quietly dropping revenue in a segment we didn't watch." Read it aloud at the kick-off. Specificity here is what unlocks honest answers later.
Step 2 — Silent individual generation, 7 minutes. Each participant writes down every reason the failure happened, one per line. No discussion, no Slack, no looking at neighbours. Silent generation is the single biggest driver of list quality — group brainstorming converges too fast on the loudest voice. Aim for 8-12 reasons per person.
Step 3 — Round-robin consolidation. Go around the room and add one reason at a time to a shared board, deduping as you go. Cluster into four buckets: tracking & measurement, technical/site-speed, hypothesis/UX, and external (seasonality, paid traffic mix, inventory). A complete pre-mortem usually surfaces 20-40 distinct risks after deduping.
Step 4 — Score each risk on two axes: likelihood (1-5) and impact-if-true (1-5). Multiply for a risk score. Anything ≥ 16 is a blocker — fix it before the test goes live, or kill the test. Scores 9-15 are monitor-and-go: define the metric, threshold, and owner who'll watch it daily. Below 9, log it and move on.
Step 5 — Assign owners and pre-commit responses. For every blocker and monitor item, name one person and write the response in advance: "If mobile bounce rate on the variant exceeds control by 8% on day 3, Maria pauses the test." Pre-commitment beats real-time judgement once the test is live and ego is invested. Then feed the surviving hypotheses into your experiment prioritization scoring — ICE, PIE, or whatever framework you already run.
Step 6 — Archive and review. Keep every pre-mortem in a searchable doc alongside the eventual test result. After 10-15 tests you'll see patterns — the same three risks keep killing your apparel category tests, or your beauty SKU launches always trip on out-of-stock variant rotation. That archive becomes the most valuable piece of operator decision-framework documentation your team owns.
Pre-mortem analysis FAQ
For a single A/B test, 30-45 minutes is the sweet spot: 5 min framing, 7 min silent generation, 15 min consolidation, 10 min scoring and ownership. For a larger launch (new checkout flow, replatform), budget 90 minutes and split scoring into a follow-up session.
The PM or CRO lead owning the test, one engineer, one analyst who knows the data layer, and at least one person who didn't help design the hypothesis — they're your best source of outside-view criticism. Keep it to 4-6 people; larger groups silence the quieter voices the exercise is designed to surface.
A risk register is a maintained list across a project. A pre-mortem is a 30-minute event that generates the list from a specific failure framing. The pre-mortem produces richer, more honest input because of the prospective-hindsight framing; the register is where you store and track what came out of it.
Skip it for low-stakes, fully reversible changes — copy tweaks on a low-traffic page, button colour tests on staging. Run it for anything that touches checkout, pricing, paid acquisition spend, or takes more than two sprint-weeks of dev. The cost is low; the cost of a botched checkout test isn't.
No — they're complementary. The pre-mortem surfaces predictable risks before launch. The post-mortem captures the unpredictable ones after. Comparing the two over time tells you whether your team is getting better at foresight, which is the actual long-term goal.
Run the pre-mortem on shortlisted hypotheses, then feed survivors into your prioritization scoring. Hypotheses with too many high-score blockers get demoted or sent back for redesign. This stops your roadmap from filling up with high-ICE-score tests that were never going to execute cleanly.
That's a signal the framing wasn't vivid enough. Re-read the failure scenario aloud with a specific dollar number attached: "We lost €45,000 in revenue over the test window." Concrete loss numbers unlock more honest answers than abstract "the test failed" framings.
No — they're too invested. Have a neutral facilitator run it, ideally someone from analytics or a different squad. The author participates as a writer, not a defender, and is explicitly told not to respond to criticism during silent generation or consolidation.
Yes — it works on anything with a defined success criterion and a meaningful cost of failure. Replatforming to a new Shopify theme, launching a subscription tier, rolling out a Klaviyo migration. The framing changes from "the test lost" to "the launch missed its target by 50%" but the mechanic is identical.
Two safeguards. First, track the kill rate: a healthy pre-mortem cadence kills or redesigns 15-25% of incoming hypotheses. If you've run twenty and killed zero, the exercise is theatre. Second, rotate the facilitator quarterly so the questioning style stays fresh.
Test ideas before you ship them
Run unlimited A/B tests, attach hypotheses to outcomes, and build a searchable archive of what works — and what doesn't.