Strength of Evidence: The Likelihood Ratio

Last lesson left one piece in a black box. In the odds form — posterior odds = prior odds × likelihood ratio — the prior odds you can usually estimate, and the multiplying is just arithmetic. But that middle term, the likelihood ratio, is the part that actually says how hard a clue should hit you. Two pieces of evidence can both “point the same way” while one nudges you a hair and the other flips your whole belief. The difference between them is entirely captured by one number, and this lesson is about reading it.

Here’s the thing nobody tells you about evidence: its strength has almost nothing to do with how confirming it feels. A clue can scream “guilty!” at your gut and be worth essentially zero, while a dull, technical detail quietly settles the case. The gut measures vividness; the likelihood ratio measures diagnosticity — and only one of those tracks the truth. By the end of this lesson you’ll be able to look at any clue and ask the one question that matters: not “does this fit my theory?” but “how much better does it fit my theory than the alternative?”

That second question is the whole game. Hold it tight; we’ll spend the rest of the lesson earning it.

One number for how strong a clue is

Formally, the strength of a piece of evidence is its likelihood ratio (LR):

$\text{LR} = \frac{P(E \mid H)}{P(E \mid \lnot H)}$

In words: how many times more likely the evidence $E$ is if your hypothesis $H$ is true than if it’s false. It’s the same likelihood ratio you met in the odds form last lesson — and it is the entire strength of a clue, packed into one number. Everything about “how much should this move me” lives here. Recap the rule you already proved: posterior odds = prior odds × LR. The prior is where you stood; the LR is how hard the new clue shoves.

The scale is wonderfully intuitive once you see where $1$ sits:

• LR = 1 → the evidence is exactly as likely whether $H$ is true or false. It’s non-diagnostic — useless. Multiply your odds by $1$ and nothing happens. (This is the “useless evidence moves nothing” case from the odds form.)
• LR > 1 → evidence for $H$. The bigger the number, the harder it points. LR = 2 doubles your odds; LR = 100 multiplies them a hundredfold.
• LR < 1 → evidence against $H$. An LR of $\tfrac{1}{10} = 0.1$ cuts your odds to a tenth. (Evidence for $H$ is automatically evidence against $\lnot H$, and vice versa.)
• LR very large or very small → strong evidence, in whichever direction.

A rough feel for the magnitudes, with a concrete clue for each:

Notice the table is symmetric in spirit: an LR of $\tfrac{1}{100}$ is exactly as strong as an LR of $100$ — it just points the other way. Strength is about distance from 1, in either direction.

Diagnostic vs. worthless: it’s about the alternative

Here is the single most important idea in the lesson, and it follows straight from the formula. The denominator $P(E \mid \lnot H)$ — how likely the evidence is under the alternative — is what makes or breaks a clue. Strong evidence must be unlikely under the alternative. If something is common whether or not your hypothesis holds, it can’t discriminate between them, and the ratio collapses toward 1.

Take the detective’s two clues from the pretest, in numbers. Suppose almost every household owns a kitchen knife — so $P(\text{owns a knife} \mid \text{guilty}) \approx 0.95$ and $P(\text{owns a knife} \mid \text{innocent}) \approx 0.90$. The likelihood ratio:

$\text{LR}_{\text{knife}} = \frac{0.95}{0.90} \approx 1.06$

Barely above 1. The “smoking gun” of owning a knife multiplies your odds of guilt by about $1.06$ — a rounding error. It feels damning because it connects the suspect to the weapon, but it’s nearly worthless, because it’s almost as likely for an innocent person.

Now the rare shoe print. Say a guilty suspect who was at the scene leaves that tread with probability $P(E \mid \text{guilty}) \approx 0.9$, while a random innocent person happens to own that rare tread with probability $P(E \mid \text{innocent}) \approx 0.002$ (fewer than 1 in 500 shoes):

$\text{LR}_{\text{print}} = \frac{0.9}{0.002} = 450$

An LR of 450. That is a clue that earns its keep — not because it fits guilt better (both clues fit guilt fine) but because it’s rare under innocence. The whole difference between a worthless clue and a decisive one lives in the denominator.

Weak clues, multiplied

A single weak clue moves you a little. But weak clues stack — and in odds form, stacking is just multiplication. If clues are independent, you chain their likelihood ratios:

$\text{posterior odds} = \text{prior odds} \times \text{LR}_1 \times \text{LR}_2 \times \text{LR}_3 \times \cdots$

Each new clue is another factor. So a string of individually unimpressive clues can multiply into a strong conclusion. Suppose you start at even odds, $1 : 1$, that a particular email is spam, and three weak signals arrive — each with a likelihood ratio of about $3$ (the word “winner,” an unfamiliar sender domain, a suspicious link):

$\underbrace{1 : 1}_{\text{prior odds}} \times 3 \times 3 \times 3 = 27 : 1$

Three forgettable clues, each barely worth a second look on its own, multiply to 27 : 1 odds — about a $\tfrac{27}{27+1} \approx 96\%$ chance of spam. This is exactly how a naive Bayes spam filter convicts an email: no single word is proof, but enough mildly suspicious words multiply past any reasonable threshold. Detectives, doctors stacking symptoms, and hiring panels combining weak signals all run the same engine.

One crucial caveat, which gets its own lesson next: the clues must be independent. If two “clues” are really the same fact wearing two hats — say, two witnesses who only heard the story from each other — multiplying both LRs double-counts the evidence and overshoots wildly. We’ll dissect that trap in lesson 5; for now, just flag that multiplication assumes the clues are genuinely separate.

Use the slider to feel evidence strength

The likelihood ratio is just the true-positive rate divided by the false-positive rate — sensitivity over the false-alarm rate. Drag those two and watch how a stronger ratio moves the posterior further from the prior. Same prior, same positive result; only the strength of the clue changes.

At the start, $\text{LR} = \tfrac{90}{45} = 2$: a positive nudges a 10% prior up to only about 18%. Drag the false-positive rate down to 9% and $\text{LR} = \tfrac{90}{9} = 10$: the same positive now carries you to roughly 53%. The prior never moved and the test still “came back positive” — all that changed is how unlikely that positive is under the alternative, which is the whole strength of the evidence.

Extraordinary claims need extraordinary evidence

This is the most practical takeaway in the course, and it’s pure Bayes. The maxim — Laplace’s, popularized by Hume, Sagan, and Truzzi as “extraordinary claims require extraordinary evidence” — isn’t a slogan about skepticism. It’s a direct reading of posterior odds = prior odds × LR. A claim that’s extraordinary has, by definition, tiny prior odds. To drag those tiny odds up to anything believable, the likelihood ratio has to be enormous — extraordinary — to compensate.

Work it through. Suppose someone claims to be genuinely psychic, and you put the prior odds of true psychic ability at about $1$ in a billion, i.e. $1 : 1{,}000{,}000{,}000$. They then pass a test that would be passed only $1\%$ of the time by a fraud — sounds impressive! That’s a likelihood ratio of about:

$\text{LR} = \frac{P(\text{passes} \mid \text{psychic})}{P(\text{passes} \mid \text{faking})} \approx \frac{1}{0.01} = 100$

An LR of 100 is genuinely strong evidence in everyday life. Run it against this prior, though:

$\underbrace{1 : 1{,}000{,}000{,}000}_{\text{prior odds}} \times 100 = 100 : 1{,}000{,}000{,}000 = 1 : 10{,}000{,}000$

After “passing,” the odds of real psychic ability are still $1$ in $10$ million. The strong-seeming clue barely dented the claim, because it was fighting a prior a billion-to-one against. To get even to even odds — $1 : 1$ — you’d need:

$\text{LR} = 1{,}000{,}000{,}000$

A likelihood ratio of about a billion. That means evidence so specific it would essentially never appear if the person were faking — $P(E \mid \text{faking})$ on the order of one in a billion. Ordinary “impressive” evidence (an LR of a few hundred) doesn’t come close. That’s the whole content of the maxim: it’s not that extraordinary claims are forbidden, it’s that the bar for the likelihood ratio scales with how small the prior is.

When to reach for it

Reach for the likelihood ratio whenever you catch yourself about to be moved by a clue — a damning detail, a glowing reference, a scary statistic, a viral claim. Before you let it shift you, run two quick checks. First, the diagnosticity check: “how likely would I see this exact thing if my hypothesis were false?” If “about as likely,” the LR is near 1 and you should ignore the clue, however vivid. Second, the extraordinariness check: for any claim with a tiny prior, ask whether the evidence is extraordinary enough — would it be near-impossible under the boring alternative? — before you let it carry you. And when several genuinely independent clues arrive, multiply their LRs to combine them. These three moves — judge by the alternative, scale the bar to the prior, multiply the independent — are the practical core of weighing evidence.

Where this goes next

You can now read a clue’s strength straight off its likelihood ratio, spot worthless evidence by checking the alternative, multiply independent clues into a strong conclusion, and see why a tiny prior demands extraordinary evidence. That’s the engine running correctly. Next, in “Where Updating Goes Wrong,” we watch it break: people who drop the prior and treat the evidence as the whole answer, the inverse fallacy of confusing $P(E \mid H)$ with $P(H \mid E)$, the double-counting of correlated “clues” that aren’t really independent (the caveat we flagged in the chaining section), and the stubborn refusal to update at all. Knowing the failure modes is what turns a formula into judgment.