In the story that gave this whole failure mode its name, British officials in colonial Delhi were worried about venomous cobras, so they offered a bounty for every dead cobra brought in. It worked — at first. Then enterprising locals did the rational thing: they started breeding cobras to kill for the bounty. When the officials discovered this and scrapped the program, the breeders, now stuck with worthless snakes, released them. Delhi ended up with more cobras than before it started paying to remove them. Economists call this the cobra effect, and it is the dark twin of everything you’ve learned so far: incentives are so powerful that pointing one at the wrong target doesn’t just fail — it actively produces the opposite of what you wanted.
You met the slogan in lesson 1 (show me the incentive) and the self-deception in lesson 3 (incentive-caused bias). This lesson is about the most expensive mistake in the whole subject: rewarding the wrong thing. It’s expensive precisely because incentives work. A weak incentive pointed at a bad target does little harm. A strong one — exactly the kind you’re proud of designing — drags everyone straight toward the bad target with full force. Let’s make you commit to a guess first.
Before you read — take a guess
A hospital is judged on one number: the percentage of emergency patients seen within four hours. Staff start parking incoming ambulances outside (so the clock doesn't start) and admitting people to random wards just before the deadline. What's the cleanest description of what went wrong?
Perverse incentives: when the reward backfires
The precise definition. A perverse incentive is a reward that produces the opposite of the intended effect — it incentivizes exactly the behavior it was meant to prevent. The cobra bounty was supposed to reduce cobras; it grew them. It’s not that the incentive was weak. It’s that it was aimed at the wrong thing and worked perfectly on that wrong thing.
Why it happens — the proxy problem. Here’s the root cause, and it’s almost unavoidable. The thing you actually care about is usually hard to measure: “good emergency care,” “real pest control,” “a healthy sales relationship,” “genuine learning.” So you reward a proxy instead — a measurable stand-in that correlates with the real goal under normal conditions. Cobras killed, tails handed in, patients-within-four-hours, accounts opened, test scores. The proxy seems fine, because before you start rewarding it, it really does track the goal.
The trouble is that a proxy correlates with the goal only until you start pushing on it. The instant a big reward lands on the proxy, people find all the ways to move the proxy that don’t move the goal — because those are almost always cheaper than actually achieving the goal. Breeding cobras is cheaper than hunting wild ones. Parking ambulances is cheaper than curing patients faster. The reward doesn’t care which path you took; it only sees the proxy. So the cheap, goal-free path wins.
The proxy trap, in one line
You can’t reward the goal directly, so you reward a measurable proxy for it. But people optimize exactly what’s rewarded — so they find the cheapest way to move the proxy, which is almost never the way that also moves the goal. The stronger the reward, the wider the gap between the two.
Why is rewarding a 'proxy' (a measurable stand-in) so prone to backfiring, even when the proxy genuinely correlated with the goal beforehand?
Watch a population defect — the Hanoi rat farms
In 1902, the French colonial administration in Hanoi had a rat problem of its own, and reached for the same tool: a bounty paid for every rat tail handed in. Tails arrived by the tens of thousands. And yet the rats kept coming — because the bounty was on tails, not dead rats. Catchers learned to snip the tail off a live rat and release it (a tailless rat still breeds), and the truly entrepreneurial started farming rats outright. The administration was paying, with great efficiency, for exactly what it asked for: tails. It just never wanted tails. It wanted fewer rats.
The slider below is that bounty. Drag the reward up and watch the population: at a low bounty, the cheap honest path (actually hunting wild rats) is good enough, so most people do the intended thing. As the bounty climbs, person after person crosses their private tipping point — the reward where gaming becomes worth more than honest work — and defects to farming. This is the cobra effect in motion: nobody’s character changed. The reward did.
Follow the reward
The Hanoi rat-tail bounty
Drag the reward up. Each actor defects to gaming the metric once the payoff clears their personal price — behavior follows the reward, not the stated goal.
Bounty paid per rat tail handed in
Reward 0¢: 0% (0/24) now choose “Farming rats for tails” over “Hunting wild rats”.
Two things are worth pausing on. First, notice there’s no villain: the people who defect last are the most scrupulous, but everyone has a price, and a strong enough reward on the wrong proxy will find it. Second, notice what the fix does. Tying the reward to the real outcome (an audited drop in the rat population) instead of the proxy (tails) doesn’t just reduce the gaming — it removes the incentive to game at all, because now the only way to earn is to actually achieve the goal. That’s the entire prescription for the final lesson, previewed in one toggle.
On the slider, the rat-catchers who keep doing honest work at the highest bounties are the ones with the highest personal 'tipping points.' What does the model imply about relying on those scrupulous people to keep a badly-aimed incentive safe?
Goodhart’s law: the measure stops measuring
This trap is so reliable it has a law. Goodhart’s law, in its pithiest form (owed to the anthropologist Marilyn Strathern): “When a measure becomes a target, it ceases to be a good measure.” The economist Charles Goodhart originally noticed it in monetary policy — the moment a central bank targeted a particular statistic, that statistic stopped behaving the way it had when no one was steering by it. The same shape appears everywhere a number gets a reward bolted to it.
A measure works as a thermometer — a passive readout that reflects reality. The error is grabbing the thermometer and using it as a thermostat — a control you push on. The instant you do, everyone with a stake starts warming the thermometer with a lighter instead of heating the room. The number goes up; the thing the number was supposed to represent does not.
Goodhart's cousins, worth knowing by name
Campbell’s law is the social-science twin: the more any quantitative indicator is used for high-stakes decisions, the more it gets corrupted and distorts the process it was meant to monitor (think high-stakes school testing). The Lucas critique is the macroeconomics version. They’re all the same insight wearing different field jackets: a metric under pressure deforms. Knowing the names lets you spot the pattern faster.
The classic illustration — possibly apocryphal but too perfect not to teach — is the Soviet nail factory. Rate the plant on the number of nails it produces and it ships millions of tiny, useless tacks. Switch to rating it on the weight of nails and it ships a single gigantic, equally useless nail. Each time, the factory perfectly maximizes the measure and completely abandons the goal (usable nails), because the measure was never the goal — just a stand-in that snapped the moment it was put under load.
No single metric is gaming-proof, because every metric is a projection of a rich goal onto one cheap number, and a projection always throws information away. Whatever the number ignores is a free lunch for anyone optimizing it: ignore quality and they cut quality; ignore the long term and they borrow from it; ignore the patients you turned away and they turn away the hard ones. The practical defenses aren’t “find the one true metric” — they’re using several metrics that are gamed in opposite directions, pairing a quantity measure with a quality guardrail, auditing for gaming, and keeping humans in the loop with the discretion to say “you hit the number but you obviously cheated.” We’ll build these into a design checklist in lesson 6.
Worked cases: the same machine, five disguises
Every one of these is one organization rewarding a proxy and getting the proxy — goal not included.
| Setting | Real goal | Rewarded proxy | What people did instead |
|---|---|---|---|
| Hanoi, 1902 | Fewer rats | Rat tails handed in | Snipped tails off live rats; farmed rats for tails |
| Wells Fargo, ~2011–2016 | Customers who want more products | New accounts opened per employee | Opened ~3.5 million accounts customers never asked for |
| UK A&E targets | Good, timely emergency care | % patients processed within 4 hours | Held ambulances outside; admitted people to hit the clock |
| High-stakes school testing | Students who’ve genuinely learned | Standardized test scores | Narrowed teaching to the test; in some districts, outright cheating |
| Surgeon “scorecards” | More lives saved | Risk-adjusted survival rate per surgeon | Avoided operating on the sickest patients to protect the stat |
Read down the “rewarded proxy” column and you can almost predict the last column before reading it. That’s the tell of a mature incentives thinker: shown a proxy with a reward on it, you can forecast the gaming in advance. The fix is never “try harder to be good.” It’s to change what’s in the middle column.
The pre-mortem question that catches most of these
Before you attach a reward to any number, ask: “If a clever, lazy, slightly dishonest person wanted to maximize this number without doing the real work, how would they do it?” Whatever you imagine, assume someone will actually do it. If the cheapest way to win the reward isn’t the same as achieving the goal, you’ve found a perverse incentive on the drawing board — which is the cheapest possible place to find one.
A content platform pays creators per 'minute watched.' Within months, videos balloon to triple their old length, padded with filler, and genuinely useful short videos vanish. Apply the lesson: what's the diagnosis, and the right class of fix?
Recap
You’ve now met the most expensive way to use the most powerful lever:
- A perverse incentive produces the opposite of what was intended — not from weakness, but from a strong reward aimed at the wrong target (the cobra effect).
- The root cause is the proxy problem: the real goal is hard to measure, so you reward a measurable stand-in — and people move the proxy by the cheapest route, which usually doesn’t move the goal.
- Goodhart’s law: when a measure becomes a target, it ceases to be a good measure. A thermometer makes a terrible thermostat. (Campbell’s law and the Lucas critique are the same insight elsewhere.)
- The same machine drives the Hanoi rats, Wells Fargo’s fake accounts, gamed hospital targets, teaching-to-the-test, and surgeons dodging hard cases — read the rewarded proxy and you can forecast the gaming.
- The defense isn’t virtue, it’s design: ask how would a clever, lazy, slightly dishonest person game this? before you ship the reward — and aim it at the outcome, not the proxy.
Check yourself: perverse incentives
What makes an incentive "perverse," as opposed to merely weak or ineffective?
Check your answer to continue.
Where this goes next
So far the story has been one-directional: incentives are powerful, and pointing them at the wrong target backfires. But there’s a stranger possibility we’ve only hinted at — that adding a reward to something people were already doing can make them do it less. The Hanoi administration at least got tails. Sometimes you pay for a behavior and watch the behavior shrink, because the very act of attaching money to it changed what it meant. That’s lesson 5, Intrinsic vs. Extrinsic — where a daycare center tries to stop late pickups with a fine and ends up with more of them, and we learn the one situation where the obvious move (just pay for it) is precisely the wrong one.