A Hundred Costumes

You now have the machinery: a proxy that tracks its goal in the calm middle of the distribution, optimisation pressure that stampedes behaviour to the proxy’s extreme, and four named flavours — regressional, extremal, causal, adversarial — for exactly how the number peels away from the truth. This lesson is the field guide. We’re going to walk a museum of famous disasters and, at each exhibit, do the same four-step autopsy: name the goal, name the proxy that got rewarded, show exactly how it was gamed, and pin the Goodhart flavour. Different century, different industry, different villain — same skeleton every time. Once you’ve seen it wearing a hundred costumes, you’ll recognise the hundred-and-first before it finishes dressing.

The archetype: one skeleton under every costume

Lesson 00 opened with the Soviet nail factory: told to hit a target in tonnes, it forged a single 70-tonne nail; retargeted on the count of nails, it stamped millions of useless tacks. Keep it as your reference skeleton, because every case below is the same three bones rearranged:

1. A goal nobody can measure directly (useful nails; public health; real learning).
2. A proxy that stood in for it and was cheap to count (tonnes of steel; dead snakes; test scores).
3. Optimisation pressure — a reward or punishment riding on the proxy — that makes hitting the number, not the goal, the entire job.

The costume is whatever industry we’re in. The skeleton never changes. Your job in this lesson is to stop being fooled by costumes.

When the metric grows an enemy: adversarial classics

The flavour most people mean by “Goodhart” is adversarial — a thinking agent who learns your metric and games it on purpose. Here it is in its purest, most spectacular forms.

The cobra effect and the Hanoi rat massacre

You met the Delhi cobras in the pretest. The mechanism is worth slowing down on, because it’s sneakier than “people cheated”. The bounty didn’t just fail to reduce cobras — it manufactured them. Paying per dead cobra created a market in which the profitable move was to farm the very pest you were trying to eliminate. That’s adversarial (people gamed it knowingly) sitting on top of causal Goodhart: handing in a dead cobra was correlated with “fewer wild cobras” only as long as the snakes came from the wild. Sever that assumption and the lever you’re pulling connects to nothing you care about.

Hanoi, 1902, ran the same experiment with rats. The French colonial administration, fighting plague through the city’s brand-new sewers, paid a bounty for each rat killed — provable by handing in a tail. Soon the streets filled with rats that had no tails: hunters snipped the tail, collected the bounty, and released the rat alive to breed more bountiable rats. Inspectors even found rat farms on the city outskirts. Same skeleton as the cobras, same flavours, a fresh costume — and a permanent lesson: if you reward a token of the goal (a tail, a corpse, a receipt) rather than the goal itself, you get a factory for tokens.

Teaching to the test, and the Atlanta erasures

Goal: children who have genuinely learned. Proxy: scores on a standardised test. As Campbell’s Law predicted, the harder those scores drive pay, funding, and headlines, the harder they rot. The mild version is teaching to the test: drilling last year’s paper, coaching answer-elimination tricks, and narrowing the curriculum until anything untested — art, history, curiosity — is quietly starved. Worse is “educational triage”: schools pour resources into the pupils sitting just below the pass threshold (where a point moves the metric most) while writing off both the safely-passing and the hopeless. The proxy climbs; the distribution of actual learning is deformed to serve it.

Then there’s the flat-out adversarial end. In the 2009 Atlanta Public Schools scandal, staff under intense score pressure held “erasure parties,” changing students’ wrong answers to right ones after the tests were collected. Investigators found suspiciously high wrong-to-right erasure rates across dozens of schools; the fallout ended careers and produced criminal convictions. Nobody’s understanding improved by a single fact — but the number did exactly what it was told.

Wells Fargo: eight is great

Goal: deepen genuine customer relationships. Proxy: “cross-sell” — the average number of products (checking, savings, credit cards) per customer, with an internal war-cry of “eight is great” (eight products per household). Employees were pressured relentlessly to hit daily cross-sell quotas. The cheapest route to the number wasn’t to win customers over — it was to open accounts they never asked for: some 3.5 million fraudulent or unauthorised accounts, complete with fake PINs and forged signatures, surfaced in the 2016 scandal. The cross-sell figure looked spectacular right up to the moment regulators, a $185 million fine, and years of litigation revealed it was measuring fraud, not loyalty. Textbook adversarial Goodhart, at industrial scale, blessed from the top.

Quota games inside the org chart

Put a self-interested employee between your metric and your goal and every flavour gets an accelerant. These are the cases where the gaming is quieter — buried in timestamps, reclassifications, and quarter-end scrambles — but no less total.

Sales quotas, sandbagging, and channel stuffing

Goal: sustainable revenue from happy customers. Proxy: the quarterly sales number against a quota. Reps optimise the timing and shape of the number rather than the underlying business. Sandbagging: deals that could close now are held back to pad next quarter (or pulled forward to rescue a weak one). Deep discounting at quarter-end buys the number by destroying the margin. And channel stuffing — shipping distributors far more product than they can sell, booking it as revenue today — inflates this quarter and guarantees a hangover next one. The revenue figure is hit precisely; the healthy, repeatable revenue it was supposed to represent is quietly cannibalised. There’s also a regressional twist: crown the single top rep of the quarter and you’ve partly selected for a lucky territory and easy deals, so next quarter they slump back toward the pack.

The A&E four-hour target

Goal: patients seen and treated promptly. Proxy: the share of emergency-department patients admitted, transferred, or discharged within four hours. This is a gorgeous specimen of extremal Goodhart: the link between “faster” and “healthier” is real across the normal range, then snaps at the target boundary, because all the optimisation pressure piles up on one arbitrary edge. Reported responses to versions of this target have included holding ambulances outside the doors so the four-hour clock never starts ticking, and a bunching of admissions and discharges in the frantic minutes just before the deadline — the patient booked out at 3 hours 59, whether or not that’s clinically the right moment. The percentage-under-four-hours gleams; whether care actually improved is a separate question the number has stopped answering.

Arrest quotas and CompStat

Goal: a safer public. Proxy: crime statistics — arrests, tickets, and recorded offences, made famous by New York’s CompStat system that grilled precinct commanders on their numbers. Two opposite games appear at once. Chasing easy collars and ticket quotas floods effort toward whatever’s cheapest to count, not whatever’s most harmful. And downgrading — reclassifying a burglary as “lost property,” discouraging victims from filing, or shelving reports — makes recorded crime fall without touching real crime. Officers surveyed anonymously described exactly this pressure. The dashboard glows green; the streets are unchanged, and sometimes the misallocated effort makes them worse.

When the optimiser is a machine: engagement

Goal: show people content they genuinely value. Proxy: measurable engagement — clicks, watch-time, sessions, shares. Here the adversarial optimiser isn’t a person breaking rules; it’s a recommendation algorithm doing exactly its job, tuned to maximise the proxy across billions of impressions. And it discovers, tirelessly, that the cheapest fuel for clicks and watch-time isn’t value — it’s clickbait headlines that overpromise, outrage and moral fury that keep thumbs stuck to the screen, and rabbit-hole recommendations that autoplay you toward ever more extreme content because extremity retains. The engagement graph goes vertical; whether anyone is better informed, calmer, or genuinely served slides the other way. It’s extremal Goodhart at planetary scale: watch-time correlated with value in the gentle middle, and the optimiser lives permanently at the edge where they’ve come apart.

When one number hides the risk: finance and academia

The subtlest costumes compress a rich, messy reality into a single reassuring number — and the danger is precisely that the number keeps looking fine while the thing it hides grows monstrous.

Value-at-Risk before 2008

Goal: understand how much a portfolio could lose. Proxy: Value-at-Risk (VaR) — a single figure claiming “we won’t lose more than X on 99% of days.” Managed to, reported on, and rewarded for, VaR became the number, and the number had a fatal blind spot: it says nothing about the size of losses in the other 1% — the tail. Positions were shaped to look calm under VaR while stuffing catastrophic risk into that unmeasured tail, and models leaned on the placid, correlation-light data of the pre-crisis years. When 2008 arrived, the tail the metric ignored swallowed the institutions relying on it. Extremal and causal at once: the risk–VaR link held in normal markets and shattered in the extreme, and optimising the number never actually reduced the underlying danger it failed to see.

Citation counts and “publish or perish”

Goal: produce important, true knowledge. Proxy: publication and citation counts — paper tallies, h-indices, journal impact factors — deciding hiring, tenure, and grants. Optimise those and a zoo of games appears: salami-slicing one finding into many “least publishable units”; citation rings where groups cite each other to pump the score; and p-hacking — torturing data until a publishable result confesses. The bibliometrics soar; the share of findings that are actually solid can fall, feeding the replication crisis. Adversarial gaming of a proxy, with a regressional sting on top: celebrate whoever has the single hottest paper this year, and you’ve partly crowned a fluke that reverts tomorrow.

The whole museum on one wall

Here is the parade in a single table. Read it column by column and the shared skeleton jumps out: every row is goal → cheap proxy → the cheapest way to move the proxy without the goal → the flavour that describes the split.

A few more costumes for the same skeleton, in one breath: Soviet factories again — chandeliers and sheet glass targeted by weight came out absurdly heavy, then targeted by area came out uselessly thin; surgeon league tables that rank by patient survival quietly push surgeons to refuse the sickest, riskiest patients so their scores stay clean; and response-time SLAs on support tickets get “met” by instantly sending a useless auto-reply that stops the clock while solving nothing.

Recap

Ten industries, one skeleton. Whether the gamer is a snake breeder, a stressed teacher, a quarter-end sales rep, an ambulance queue, a recommendation algorithm, or a whole bank, the move is identical: reward a proxy hard enough and someone — or something — finds the cheapest path to the number that skips the goal. You can now run the four-step autopsy on any KPI you meet. The natural next question is why this keeps happening — is it bad luck, or is Goodhart the guaranteed output of a deeper machine? Next lesson we open that machine: incentives, feedback loops, and the principal–agent problem that make Goodhart not an accident but a prediction.