The intro gave you the bombers and the one-sentence model. This lesson takes the model apart to see the gears turn. Because underneath every version of the trap — the funds, the dropouts, the sturdy old houses — is a single mechanic, and once you can name its three moving parts you’ll spot it going off in the wild. Let’s watch a filter delete half a dataset in slow motion.
Before you read — take a guess
A jobs site brags: 'People who completed our coding bootcamp earn an average of $95,000.' They surveyed graduates who are currently employed as developers. What is most likely inflating that number?
Population, filter, survivors: the three moving parts
Picture a coffee filter. You pour in ground beans and water — that’s everything that went in. The filter holds back the grounds. What lands in your cup is smooth, dark, and grounds-free — and if you judged the beans by the cup, you’d swear coffee grounds don’t exist. The cup isn’t lying. It’s just not the whole pour.
Survivorship bias runs on exactly three parts:
- The population — everything that entered the process. Every startup founded, every plane that flew the mission, every bean that went in.
- The filter — the mechanism that removes some of them. Bankruptcy, being shot down, the grounds caught in the paper.
- The survivors — what’s left to be observed afterward. The companies still trading, the planes that landed, the coffee in the cup.
The bias itself has a one-line definition worth memorising:
The core mechanic
Survivorship bias is mistaking the survivors for the population — reading traits, averages, or lessons off the filtered remainder as if no filter ever ran.
The trap isn’t that survivors are weird. It’s that the filter chose them, so anything you measure among them is entangled with “whatever it took to get through the filter.” Measure the wrong thing and you don’t get a slightly-off answer — you get an answer bent in a predictable direction.
A worked example: the average that quadruples out of nowhere
Say 1,000 startups are founded in a given year, each hoping to be the next big thing. Run the clock forward five years. Suppose the filter — call it “still in business after five years” — leaves 100 survivors, and the other 900 have folded to roughly zero revenue. Now a reporter wants “the average revenue of a company from that cohort.” Watch what happens depending on who they can find to ask.
| Companies counted | Total revenue | Average revenue | |
|---|---|---|---|
| Survivors only (the 100 still trading) | 100 | $200,000,000 | $2,000,000 |
| Dead companies (the 900 that folded) | 900 | $0 | $0 |
| True population (all 1,000 founded) | 1,000 | $200,000,000 | $200,000 |
Same underlying reality, two answers that differ by 10×. The survivor-only average says a company from this cohort is worth $2,000,000 a year. The honest population average — which includes the 900 corpses at $0 — is $200,000. The reporter didn’t do bad arithmetic. They divided the right total by the wrong denominator: 100 instead of 1,000. The 900 failures didn’t lower the average because they were never in the sum.
The denominator is where the bias hides
Almost every survivorship error is a division by the survivor count when it should be a division by the population count. When someone quotes an average of “companies / funds / graduates / marriages that lasted”, the first question is always: how many started, and where did they go?
Using the startup numbers above, which statement correctly names the mistake in the $2,000,000 figure?
Silent evidence: why the data looks complete
Here’s the property that makes this bias so much nastier than ordinary missing data. When a spreadsheet has a blank cell, you notice. A gap in a chart, an empty column, a “N/A” — your eye catches it and you go find the number. Survivorship bias never gives you that courtesy. The failures aren’t recorded as blanks; the rows are gone entirely. The dataset looks full, tidy, complete — a table with no holes in it — precisely because the holes were deleted along with everything in them.
Nassim Taleb calls this silent evidence: the data that would have changed your mind, sitting in a graveyard you never think to visit. It doesn’t argue with you. It doesn’t show up as a suspicious gap. It just isn’t there, and its absence is invisible.
| Kind of missing data | What you see | Do you notice? |
|---|---|---|
| Obvious gap | A blank cell, “N/A”, a hole in the chart | Yes — you go find the value |
| Silent evidence (survivorship) | A clean, complete-looking table of survivors | No — the failures were deleted with their rows |
That’s the difference between a witness who refuses to answer and a witness who was quietly removed from the courthouse. The first makes you suspicious. The second lets the trial proceed as if they never existed.
An investing site lists every mutual fund available to buy today and reports their average 10-year return: a cheerful 9%. You buy the whole list, believing 9% is what “funds” return. Predict: why is the real average return of all funds that existed 10 years ago lower than 9% — and where did the difference go?
The answer: funds that performed badly over the decade were closed, merged away, or liquidated, and dropped out of the “available today” list entirely. That practice has a name — survivorship bias in fund databases — and it inflates reported industry returns by roughly 1–1.5 percentage points a year. The losers aren’t marked with an asterisk; their rows are simply gone. You’re reading a leaderboard with every loser quietly erased, then mistaking it for the game.
Diagoras and the drowned worshippers
The oldest recorded version of this argument is about 2,400 years old, and Taleb loves to retell it. Cicero passes down the story of Diagoras of Melos, nicknamed “the Atheist.”
Someone eager to prove that the gods answer prayers walks Diagoras through a temple and points to the wall. It’s covered in votive paintings donated by sailors who had prayed during a shipwreck and survived — plank by plank, wave by wave, saved. There, says the believer. Proof that prayer works. Look how many were spared.
Diagoras looks at the wall of grateful survivors and asks the question that ends the argument:
“But where are the pictures of those who prayed — and drowned?”
They don’t exist, of course. The drowned don’t commission paintings. They don’t hang anything on the wall. They are the silent evidence at the bottom of the sea, and the temple wall is a survivors-only sample dressed up as a complete record of what prayer does. Every painting is a real survivor; the wall is a lie of omission. Swap “prayer and shipwrecks” for “morning routines and billionaires” and you have the exact same wall, 2,400 years later.
Map each piece of the mechanic onto Diagoras's temple.
Pick a term, then click the phrase it corresponds to in the story.
The filter can be anything
The shipwreck is dramatic, but the filter doesn’t have to be deadly. Anything that removes part of the population before you observe it does the same job. A few disguises:
- Death. The soldiers who skipped the helmet and died aren’t around to say it was a bad idea.
- Bankruptcy / shutdown. Failed companies stop filing reports and drop out of the data.
- Deletion from a database. Liquidated funds are scrubbed from “available funds” lists.
- Dropping out. Students who quit the program aren’t in the graduation stats.
- Editorial selection. Magazines profile the entrepreneurs who made it; the rest never get written up.
- Self-selection. The customers who agree to be interviewed are rarely the ones who quietly left in disgust.
Notice the range. Some filters are catastrophic — a shipwreck deletes most of the people who went in. Some are mild — a survey that loses a handful of grumpy non-responders barely distorts anything. The rule of thumb:
Filter strength sets the size of the lie
The stronger the filter, the more misleading the survivors. A deadly filter (most of the population deleted) makes the survivors wildly unrepresentative; a gentle filter (a few dropouts) tilts them only slightly. Before trusting a survivor sample, estimate how hard the filter squeezed.
For each claim, is the sample you're being shown the SURVIVORS ONLY, or does it already include the FULL POPULATION?
Sort each scenario by whether a filter has already deleted part of the data.
- Average lifespan computed from every death certificate filed in a country last year
- 'Startups are wildly profitable' — from the revenue of companies that made it to their IPO
- 'This diet works' — from testimonials by people who stuck with it and reached their goal
- 'The average novel earns its author $X' — measured from books currently in print and selling
- 'Old cars were built better' — judged from the classic cars still on the road at car shows
- A factory weighs every single item off the line, defective and perfect alike, before any are removed
When to reach for it: the reflex
You don’t need to run the full analysis every time. You need one trigger to fire in your head, and it’s phonetically simple: “still.”
Any time your data is described as the ones still here, still trading, still talking, still married, still standing — a filter ran before you got to look. That word “still” is the fingerprint of a deletion. When it shows up, do one thing:
The move: reconstruct the population
When you catch yourself reasoning from survivors, rebuild the population that existed before the filter. Ask: How many started? What removed some of them? Where did the deleted ones go? Then recompute with the failures put back in — the honest denominator almost always changes the conclusion.
This is the same muscle Wald used on the bombers: he refused to reason from the returning planes alone and mentally reconstructed the whole fleet that took off, including the ones that didn’t come back. Same reflex, whether the filter is anti-aircraft fire, fund liquidation, or a shipwreck.
Fill in the mechanic.
Pick the right option for each blank, then check.
Everything that enters a process is the . The mechanism that removes some of it is the . What remains to be observed are the . Mistaking that remainder for the whole is survivorship bias, and its worst feature is that the deleted failures leave no visible gap — Taleb calls them evidence.
Recap in three questions
A charity reports that families who received its help have a 90% 'success' rate five years on. It could only locate families still reachable at the same address. Why might 90% overstate the program's effect?
Check your answer to continue.
What’s next
You now have the machine in your hands: population → filter → survivors, plus the reason it hides — silent evidence leaves no gap to notice. Every remaining lesson is this mechanic wearing a different costume.
Up next, lesson 2: Wald’s Bombers, in Full — where we run the entire calculation behind armouring the gaps. You’ll see exactly why the survivor damage map is the mirror image of the danger map, how hit rate and lethality combine, and how one reframing turned a fatal instinct into a fleet-saving decision.