Inadequate Equilibria

If the world was created by the Invisible Hand, who is good, how did it come to contain so much that is evil?

Eliezer Yudkowsky's Inadequate Equilibria, as explained through Scott Alexander's review, is an exercise in economic theodicy — a systematic attempt to explain why so many systems are obviously broken even though the logic of competition says they shouldn't be.¹ The market economy is very good at exploiting money-making opportunities. So if you see a $20 bill lying on the sidewalk of Grand Central Station, and you remember seeing the same bill a week ago, something is deeply wrong. Thousands of people cross Grand Central every week. There's no way they all missed a free $20.

Extend this logic far enough and you prove that nothing can ever be bad. Science can't be broken, because someone would set up better science and win all the grants. Health care can't be overpriced, because someone would build a cheaper hospital and steal all the patients. But babies are dying from the wrong lipids in their IV fluid, and the Bank of Japan did leave a trillion-dollar bill on the floor for a decade. So where does the logic break down?

The Three Mechanisms

Yudkowsky identifies three categories of civilizational failure, each corresponding to a different way the $20 can stay on the ground.¹

Mechanism one: no exploitable correction. If you notice a mistake but there's no way to profit from correcting it, the mistake persists. You can't short the housing market (well, you can now, but for decades you couldn't), so housing bubbles inflate uncorrected even though 90% of analysts see the crash coming — the 10% who don't just keep bidding against each other. You can't short startup valuations, so a stupid Uber-for-puppies company gets funded whenever one clueless investor likes the pitch. The Bank of Japan's policies were set by bureaucrats whose personal payoff matrix rewarded caution over correctness. Even when the correct policy was common knowledge among outsiders, nobody had a mechanism to translate that knowledge into action.

The payoff matrix for the Japanese central bankers was stark: low money supply that collapses the economy is neutral (you were prudent), while high money supply that collapses the economy is catastrophic for your reputation (you were reckless). There's no career upside to being right in a bold way, only career downside to being wrong. It's more Moloch than stupidity — individually rational choices producing collectively insane outcomes.

Mechanism two: expertise can't trickle down. The stock market stays efficient because expertise brings power: when Warren Buffett proves good at picking stocks, everyone gives him their money to invest, which lets him buy enough of an underpriced asset to correct its price. But now imagine a law saying nobody can invest more than $1000. The experts invest their $1000, and then what? The lemons problem kicks in: fraudulent experts claim to be just as good as real ones, and nobody can tell the difference.

This is medicine. The five doctors who actually understand infant nutrition can yell about how the wrong lipids are killing babies. But everyone is yelling about something. There's an inexploitable market in indignation. The dead-babies problem can't compete because it doesn't have an optimal indignation profile: no single villain, too much biochemistry, not enough virality. The people with real knowledge are drowned out by a sea of people who also claim to have real knowledge, and there's no mechanism — no equivalent of "give the expert all the money and let them correct the price" — to sort signal from noise.

Mechanism three: bad Nash equilibria. Everyone hates Facebook but nobody leaves because all their friends are there. Everyone hates the status quo but nobody defects because unilateral defection is worse than staying. These are the traps described in Moloch, and they operate at every scale. The Tower metaphor from the book — a magical tower that drains four years of your life, but employers demand proof you survived it — is a thinly veiled description of higher education. Even if a better, cheaper Tower Two appears, it can't compete because Tower One already has the signaling monopoly, and neither students nor employers can unilaterally switch.

The interaction between mechanisms is what makes things truly hopeless. A researcher who wants to work on high-impact but uncitable problems needs both a willing researcher (mechanism two) AND a willing grantmaker (mechanism one). Even if you add altruistic researchers to the model, they can't find funders. Even if you add altruistic funders, they can't find researchers. One systemic problem can often be overcome by one altruist in the right place. Two systemic problems are another matter entirely.

Inside vs. Outside View

The second half of the book, which Alexander calls "really a different book," argues against reflexive deference to expert consensus.¹ The Outside View says: "I think I've spotted an error, but most people who think they've spotted errors are wrong, so I'm probably wrong." This is often sound reasoning. Most people who think they're above-average drivers aren't. Most people impressed by time-share presentations are being duped. Most amateur scientists who think they've disproven Einstein are cranks.

But Yudkowsky points out that the Outside View can be paralyzing. If you always defer to the consensus on the grounds that contrarians are usually wrong, you can never update on object-level evidence. You end up believing the Bank of Japan's policies are correct because the Bank of Japan set them, and dismissing the evidence of your own eyes. You end up believing the infant nutrition problem doesn't exist because the FDA hasn't fixed it, which assumes the FDA's decisions are the output of a reliable epistemic process rather than a Nash equilibrium nobody can escape.

The resolution is that modesty should be calibrated to the mechanism. In highly efficient markets (the stock market), the Outside View is very strong — you probably can't beat Goldman Sachs. In markets that are efficient but not at the thing you care about (the indignation market), the Outside View is weaker. In systems broken by Nash equilibria or misaligned incentives, the Outside View can be actively misleading, because the fact that nobody has fixed the problem is itself explained by the mechanism, not by the absence of a problem.

This connects to Calibration And Measurement. Good calibration means knowing when you're in an efficient market and when you're not. The trick isn't universal confidence or universal deference — it's correctly classifying which type of system you're looking at and adjusting your credence accordingly.

The Dead Babies Are Still Dead

The most striking thing about Inadequate Equilibria is that the infant nutrition problem wasn't hypothetical. The wrong-lipid IV fluid was killing about a third of the babies who received it. The correct formulation was known, cheap, and available in other countries. The problem persisted for decades. As of the book's publication, those babies were still dying.

This should be genuinely shocking. Not because the world contains suffering — that's not news — but because this particular suffering exists inside a highly legible, well-understood system with a known fix of zero marginal cost, and still persists. It's a proof that the Invisible Hand is neither invisible nor a hand. It's a set of incentive structures that sometimes work beautifully and sometimes stand by while babies die because the FDA won't update a formula and the indignation market is saturated.

The value of understanding inadequate equilibria isn't just that it explains failure. It's that it tells you where to look for low-hanging fruit — not in efficient markets where the fruit has been picked, but in the specific niches where mechanisms one, two, or three prevent correction. Those niches are more common than the efficient-market hypothesis would have you believe.