You’ve met survivorship bias on a bomber’s fuselage and traced its skeleton in the abstract: a filter runs, the failures vanish, and the survivors lie to you by omission. Now for the payoff — because once you’re carrying this X-ray, you start seeing the same skeleton everywhere. Fund brochures. Airport-bookstore business bestsellers. Your uncle’s opinion about old houses. The playlist of “timeless” classics.
Every case below gets the same three-step X-ray. Who got filtered out? How does the survivor sample differ from the whole cohort? And which way does that flip the conclusion? Same tool, five wildly different disguises.
Before you read — take a guess
A finance site brags that 'the average actively managed stock fund returned about 9% a year over the last decade.' Funds that performed so badly they were shut down or merged away are no longer in the database being averaged. What does that do to the headline number?
Case 1: Mutual funds — the leaderboard with the losers erased
This is survivorship bias’s ancestral home. In investing the filter is almost comically literal: a fund that performs badly enough gets closed or merged into a healthier fund, and when it dies, its track record is frequently dropped from the database that analysts average to compute “how funds do.” The dogs get buried, then the vet reports that all his patients are healthy.
There’s a second, sneakier version called fund incubation: a company quietly launches ten small funds, waits a couple of years, then publicly promotes only the two or three that happened to do well and closes the rest. You’re shown the winners of a race you never knew was being run.
Here’s a clean illustration. Suppose a fund family starts the decade with 100 funds. Over ten years, 40 of them do so badly they’re shut down or merged away. If you only average the 60 survivors, the picture is rosy. Include the dead — many of which limped along at big losses before folding — and the real cohort average sinks.
| Funds counted | Average annual return | |
|---|---|---|
| What the brochure shows (survivors only) | 60 | +9.0% |
| The dead funds (closed / merged, dropped from the DB) | 40 | roughly −2% (several near −100% before folding) |
| The whole cohort (what you could actually have picked) | 100 | roughly +4.6% |
The survivors say “+9%.” The honest number for someone choosing a fund at the start — before knowing which would survive — is far lower. Real studies of this “survivorship bias in returns” estimate it inflates historical fund performance by roughly 1 to 1.5 percentage points per year. That sounds tiny until you remember compounding: 1.5 points a year, over 30 years, is a chunk of your retirement quietly evaporating between the brochure and reality.
The X-ray on funds
The filter: bad funds are closed, merged, and deleted from the record. The survivor sample: only funds healthy enough to still exist. The flip: “funds average +9%” becomes “the funds that survived averaged +9%, and you had no way to pick them in advance.”
Two databases track the same fund universe. Database A includes every fund that existed at the start of the decade, keeping dead funds' records. Database B lists only funds still open today. You compute the average 10-year return from each. What do you expect?
Case 2: “The habits of successful people”
Open any productivity feed. Billionaires wake at 5 a.m. Founders take cold showers. Everyone “followed their passion” and “never gave up.” And of course the sacred trio of college dropouts — Gates, Jobs, Zuckerberg — proof that quitting school is a launchpad.
Run the X-ray. The people who woke at 5 a.m., took the cold shower, followed the passion, and dropped out — and then failed — are not on your feed. They aren’t giving TED talks. They’re the enormous, invisible denominator. What you’re reading isn’t a list of things that cause success; it’s a list of things winners happen to share, which is a completely different thing when the losers did all of it too.
The correct question is never “what do successful people do?” It’s a comparison:
| The question the feed answers | The question that actually matters |
|---|---|
| What fraction of winners had the habit? | Of everyone with the habit, what fraction won? |
| (Look only at survivors) | …versus of everyone without it, what fraction won? |
Dropping out is the sharpest example. Yes, three of the most famous founders alive dropped out — but so did millions of people who then struggled, and the base rate of success among all dropouts is far lower than among graduates. Gates and Zuckerberg didn’t leave school to wing it; they left Harvard with a specific, already-working idea in hand. The habit (“drop out”) isn’t the cause; it rode along with a hundred other advantages. We’ll build this “find the denominator” defence properly in a later lesson — for now just notice the shape.
Winner traits are not winning causes
Any advice of the form “successful people all do X, so do X” has looked at survivors only. Before you copy the habit, ask: did the people who failed also do X? If they did, X predicts nothing.
Case 3: Business bestsellers — reverse-engineering the winners
There’s a whole genre of business book — the In Search of Excellence / Built to Last template — with an irresistible method: find a handful of companies that became spectacularly great, study them to death, extract the traits they share, and sell those traits as the secrets of greatness.
Spot the missing piece? There’s no control group. To know whether “a strong culture” or “big hairy audacious goals” cause greatness, you’d need to check companies that had the exact same traits and stayed mediocre or died. Those companies exist by the thousand — but they’re not in the book, because the book started by selecting on the outcome. Without them, the celebrated “secrets” can’t be distinguished from noise: a trait that great and failed companies share equally is worthless as a predictor, no matter how inspiring it sounds.
The punchline writes itself. Many companies crowned “excellent” or “built to last” went on to stumble, shrink, or vanish in the years after publication. That’s not always bad luck — it’s a signature of measuring survivors: if you pick this year’s winners partly because of a lucky streak, next year’s regression to the mean does the rest.
A researcher wants to know whether “visionary leadership” causes companies to become great. The bestseller studied 18 great companies and found they all had visionary leaders.
Before you peek: what’s the one dataset the book is missing — and what would it show?
The missing dataset is the comparison group: companies that also had visionary leadership but never became great (or went bankrupt). If visionary leadership is just as common among the failures, then it’s noise — it can’t be what separates great from not-great, because both groups have it. Only by comparing winners and losers with the same trait can you tell a cause from a coincidence. The bestseller studied one column of a two-by-two table and sold you the whole grid.
Case 4: “They don’t make ‘em like they used to”
Your uncle is right that the old stone house on his street has stood for 120 years. He’s wrong about what it proves.
Think about every building, bridge, chair, and tool made in 1900. The vast majority were flimsy, cheap, or unremarkable — and they rotted, collapsed, broke, or got demolished decades ago. What physically survives to the present is a heavily filtered sample: the sturdiest, the most beautiful, the most lovingly maintained. When you walk through an old town and marvel at the craftsmanship, you’re touring a survivor museum, not a fair sample of the past. The junk of 1900 is landfill; you never see it.
| What you see today | The whole 1900 cohort |
|---|---|
| Elegant, solid old buildings still standing | Plus the flimsy majority that collapsed or was demolished |
| Heirloom furniture that lasted a century | Plus the cheap stuff that fell apart in a decade |
| ”Timeless” songs still on the radio | Plus the millions of forgettable tracks nobody replays |
The music version is exactly the same trick. “Music was better back then” — but oldies radio and streaming “classics” playlists only ever replay the hits. The thousands of mediocre songs from that same year quietly disappeared. You’re comparing the best, survivor-filtered slice of the past against the entire unfiltered present, hits and flops together. It’s a rigged contest. The past isn’t better; it’s just been edited.
The tell
Whenever “the old X was better,” ask: am I seeing all of the old X, or only the pieces good enough to survive to today? The filter of time is patient and ruthless, and it only keeps the winners.
Case 5: Testimony from the ones who came back
The most seductive survivorship trap is a first-person story, because it feels like airtight evidence: I was there.
“I walked away from a 90 mph crash without a seatbelt — cars are safe / seatbelts are overrated.” The problem is who isn’t around to give the opposite testimony. The people whose unbelted crash killed them can’t tell you it didn’t work out. You’re sampling exclusively from survivors, so the advice is built on the one group guaranteed to have survived. Same energy as Titanic survivors offering survival tips — useful only if you also interview the ~1,500 who didn’t make it, which you can’t.
The strangest and most famous example comes from veterinary medicine: the cat “high-rise syndrome” data. A 1987 study of cats brought to a New York animal hospital after falling from buildings found something bizarre — cats that fell from higher floors (above about the 7th) appeared, on average, to be less injured than cats that fell from middling heights. It launched a legend that cats have some aerodynamic superpower past a certain height.
Hedge honestly: this one is debated
There may be a real physical effect (a falling cat reaches terminal velocity, relaxes, and spreads out like a parachute). But the data is a textbook survivor sample: the study only counted cats that were brought to the vet. A cat that fell from the 30th floor and died on impact was, quite often, never carried in for treatment — so the deadliest high-floor outcomes are simply missing from the dataset. The apparent “cats survive tall falls better” pattern is at least partly the downed-bomber effect with whiskers: you’re only looking at the ones that came back. Treat the aerodynamic story as possible, the survivorship artifact as near-certain.
That last case is the whole course in one image. The vets, like the 1943 engineers, were staring at a beautiful, clean pattern in their data — and the pattern was carved by what wasn’t in the data. Different species, same missing planes.
Sort the claims
Run the X-ray yourself. Some of these claims rest on a survivor-filtered sample; others come from a sample where nobody was deleted for failing.
Which claims are distorted by survivorship bias, and which rest on a clean, unfiltered sample?
Place each item in the right group.
- The average return of funds still open today is 9%, so funds are a great bet
- A pollster surveyed 2,000 randomly chosen adults, reached everyone selected, and reported the results
- Successful founders say they followed their passion, so following your passion leads to success
- Great companies in this bestseller all had bold visions, so bold visions make companies great
- A factory weighed every single part that came off the line today, defective ones included
- Old buildings are so well built — look how many centuries-old ones are still standing
- A hospital tracked outcomes for all 500 patients enrolled in the trial, including those who dropped out or died
Match each domain to its filter
The disguise changes; the mechanism doesn’t. Match each domain to the specific act that deletes its failures before you get to look.
Match each domain to the exact filter that erases its failures.
Pick a term, then click its definition.
The one move that beats all five
Notice you used the same question every single time: what got filtered out before I was allowed to look? Funds that closed, founders who flopped, companies that died, buildings that rotted, cats that never reached the vet. Five costumes, one villain. The model isn’t a fact to memorise; it’s a reflex to install — a little alarm that goes off whenever someone shows you the winners and asks you to reverse-engineer their secret.
A stock-picking newsletter advertises: 'Our recommended portfolio has beaten the market for 8 straight years!' What single fact would most undermine this claim?
Check your answer to continue.
Where this leaves you
Once installed, this reflex is almost annoying — you’ll catch it in ads, in advice, in your own head. Good. That irritation is the model working. Every “look at these winners” now triggers the counter-question, and the invisible denominator swims into view.
But there’s a nagging puzzle. If the trap is this simple to state — you’re only seeing the survivors — why does even smart, careful people’s judgement fall for it again and again? Why doesn’t an alarm ring on its own when the losers go missing? That’s exactly where we head next.
Next up: lesson 4, “Why We Fall for It” — the cognitive machinery that makes survivorship bias feel like solid evidence: winners are vivid and available, absent data rings no alarm, and our hunger for tidy causal stories does the rest.