Goodnight Wiki / Cognitive Biases

Cognitive Biases

We are not broken machines. We are machines that were built for a world we no longer inhabit, running heuristics that were tuned for ancestral environments and now misfire in grocery stores, boardrooms, and comment sections. The interesting question isn't "why are we biased?" — it's "which biases are actually bugs, and which are features we've misidentified?"

Munger's 25 Tendencies

Charlie Munger's 1995 Harvard speech, later expanded into "The Psychology of Human Misjudgment," is one of the best single-source catalogues of cognitive biases because it comes from a practitioner, not a theorist.1 Munger didn't just read about biases — he made billions navigating them in real time.

His list of 25 tendencies reads like a field guide to human firmware. Some highlights: Reward and Punishment Super-Response Tendency explains why FedEx couldn't get night shift workers to load planes on time until they switched from hourly pay to per-shift pay (incentivize what you actually want). Doubt Avoidance Tendency explains why our brains lunge at conclusions — a prey animal that deliberates gets eaten. Inconsistency Avoidance Tendency explains why habits are so hard to break, and why Einstein's willingness to destroy his own ideas made him unusual.

But Munger's real insight is about interactions. Individual biases are manageable. When they combine, they produce what he calls "lollapalooza effects" — cascading failures where multiple tendencies reinforce each other. The FedEx incentive problem involved not just reward super-response but also social proof (everyone else is going slow), authority (management says work the full shift), and doubt avoidance (why question the system?). Fix the incentive, and the whole cascade unwinds.

The Availability-Misweighing Tendency deserves special attention. Your brain doesn't estimate frequency by consulting a database — it estimates frequency by how easily examples come to mind. This is why plane crashes feel more dangerous than car rides, and why a single vivid anecdote outweighs a dry statistic. Munger's antidote is Darwinian: force yourself to seek disconfirming evidence, use checklists, and remember that "an idea or a fact is not worth more merely because it is easily available to you."1

Social-Proof Tendency is the one that should scare you most in institutional settings. Inaction by others becomes social proof that inaction is right. This is how an entire organization can watch a slow-motion disaster unfold with nobody pulling the emergency brake — not because everyone is stupid, but because everyone is looking at everyone else's inaction and concluding that inaction must be correct.

Are Sunk Costs Even a Fallacy?

The sunk cost fallacy is gospel in economics: don't throw good money after bad, past investments shouldn't affect future decisions, give up your hopes for a better yesterday. Gwern's extended analysis asks the uncomfortable question: is this actually true for individuals?2

The evidence is more complicated than the textbooks admit. Animals don't appear to commit sunk cost fallacy at all — mother mice defend litters based on current litter size, not past investment. Children commit it less than adults. And when Arkes and Blumer ran their famous theater ticket study (people who paid full price attended more plays), the effect only showed up in the first half of the season and vanished by the second half. Economics training doesn't help — even after being taught about sunk costs, people still honor them.2

Here's the revisionist case: what if sunk cost behavior is a learning heuristic? Hoeffler's evolutionary model suggests that persisting with past choices trades some expected utility for faster learning. If you constantly abandon projects when something shinier appears, you never get deep enough into anything to learn whether it works. The "nearly completed" effect dominates pure sunk cost — people persist more when they're close to finishing, which is often rational. Sunk cost avoidance doesn't even correlate with intelligence. And anecdotally, many people suffer from the opposite problem: abandoning things too quickly, not sticking with them too long.2

The real lesson may be that the ur-bias isn't sunk cost — it's thoughtlessness. People who are explicitly reminded of continuing costs don't commit the fallacy. The problem isn't that our cognitive architecture honors sunk costs; it's that we often fail to think clearly about what continuing actually costs us. This is System 1 running on autopilot when System 2 should be at the wheel.2

This connects to a deeper theme in Willpower And Akrasia: if sunk costs help counteract hyperbolic discounting — if they keep your impulsive near-self from abandoning projects that your deliberate far-self chose — then "fixing" sunk cost bias without fixing hyperbolic discounting might actually make you worse off. One bias can be medicine for another.

Cultural Transmission as Bias Correction

Joseph Henrich's The Secret of Our Success, reviewed by Scott Alexander, offers a radical reframing: what if culture is rationality, just operating on a different timescale?3

The argument begins with a puzzling fact: when European explorers were stranded in unfamiliar environments, they routinely starved to death in areas where indigenous peoples thrived. These weren't stupid people — many were from Victorian Britain, selected for education and intelligence. They failed because individual intelligence is not sufficient to reinvent the accumulated cultural knowledge needed to survive in a specific environment.

Consider Inuit seal hunting: finding breathing holes in ice, using caribou antler to assess hole shape, fashioning harpoons from polar bear bone, rendering blubber for soapstone lamps, identifying old sea ice by color for drinking water. No one person invented this. No genius reasoned it out from first principles. It accumulated through cultural evolution — trial and error, imitation, and selection operating over generations.3

This matters for cognitive biases because it suggests that much of what we call "bias" in individual cognition is actually a feature of cultural transmission. Children have hard-coded learning slots — for language, for local animals, for edible plants, for gender roles — that they fill by imitating high-prestige individuals broadly, not just in the prestige-granting domain. When an NBA player's shoe choice gets copied by aspiring basketball players, that looks like a bias. But the "if in doubt, copy it" heuristic exists because the causal links between someone's practices and their success are genuinely opaque. You can't tell which of a successful person's habits are load-bearing, so you copy broadly and let selection sort it out.3

The nixtamalization story is the clincher. Ancient Mexicans discovered that soaking corn in a solution of ground burnt seashells prevented vitamin deficiency. When conquistadors took over, they ignored this practice and ate corn straight. Three million Americans developed pellagra over four centuries, and up to a hundred thousand died, before Western science caught up in 1937. The lesson: cultural practices that look arbitrary — "superstitious," even — may encode hard-won knowledge that no individual intelligence could easily rederive.3

This is a humbling frame for the whole enterprise of debiasing. Some of what looks like Bayesian Epistemology failure at the individual level is actually the system working as designed — preserving and transmitting knowledge that no single reasoner could justify from scratch. The Bayesian prior enforcement module in your brain (see Bayesian Epistemology) may be evolutionarily tuned not to treat all evidence equally, but to strongly weight culturally transmitted information, because that information has already survived a selection process more rigorous than any individual's lifetime of evidence-gathering.

Reason as Social Technology

The most radical challenge to the standard debiasing project comes from Hugo Mercier and Dan Sperber's The Enigma of Reason: what if reason didn't evolve to help us think straight at all?4

Their argument is evolutionary. Humans' biggest advantage is cooperation, and cooperation's biggest problem is freeloading. Reason evolved not to solve abstract logical problems but to solve the social problems posed by collaborative living — specifically, to produce arguments that persuade others and to evaluate others' arguments for manipulation. This reframes confirmation bias from a bug to a feature. In Mercier and Sperber's telling, we're not biased because our reasoning hardware is broken. We're biased because our reasoning hardware is debating hardware. It's optimized for winning arguments, not for finding truth.4

The evidence is suggestive. People are much better at spotting flaws in others' arguments than in their own. When experimenters tricked participants into thinking their own previous answers were someone else's, nearly 60% rejected the responses they'd earlier been satisfied with. We're excellent at evaluating reasoning — just not our own. Mercier and Sperber call this "myside bias" rather than confirmation bias, because it's not random credulity. It's the systematic inability to turn our argumentative weapons on ourselves.4

Sloman and Fernbach's The Knowledge Illusion adds another piece: we systematically overestimate how much we understand because we can't tell where our knowledge ends and others' begins. Ask people to rate their understanding of toilets, then ask them to explain how a toilet works, and their self-assessments plummet. We're running on shared cognitive infrastructure and mistaking it for individual competence. This is normally adaptive (we should rely on distributed expertise — that's how civilization works), but it becomes dangerous in political domains where strong feelings form without deep understanding, and then get reinforced through social agreement into a confidence that has no epistemic foundation.4

The Gormans add a physiological wrinkle: people experience genuine pleasure — a dopamine hit — when processing information that confirms their beliefs. "It feels good to stick to our guns even if we are wrong." If confirmation bias has a neurochemical reward, that explains why providing people with accurate information doesn't reliably change their minds. You're fighting brain chemistry, not just faulty logic.4

Different Worlds

Perhaps the most unsettling implication of perceptual variation is that people can live in genuinely different experiential realities while sharing the same physical environment. Scott Alexander's "Different Worlds" argues that social experience is underdetermined in the same way that Bayesian perception is underdetermined — your priors shape what you encounter, and what you encounter confirms your priors.5

The clinical extremes are stark. Paranoid schizophrenics interpret every ambiguous social signal as hostile. People with Williams Syndrome — a condition involving pathological trust — will get into a stranger's car without hesitation. Between these poles, there's enormous healthy variation in how people parse the same ambiguous social world. A friend breaking two appointments might mean "something came up" or "the friendship is dying," and which interpretation feels natural depends on where you sit on the trust spectrum.5

But it goes beyond interpretation. Alexander noticed that he was unconsciously generating different data than his colleagues. His psychiatric patients never had dramatic emotional meltdowns; his colleague's patients had them constantly. A supervisor eventually pointed out that Alexander was projecting a "calm and rational" social field that made patients modulate their behavior — while his extroverted colleague's warmth invited emotional expression. Neither was wrong. They were creating different experimental conditions by existing differently in the room.5

This has disorienting implications for any debate about social experience. Some women in tech report constant discrimination; others in identical roles report none. Some black Southerners describe lifelong racism; others describe never once experiencing it. These aren't lying or deluded. They're experiencing genuinely different worlds, generated by some combination of perceptual priors, social signaling, and self-selection into different bubbles — mechanisms that are mostly invisible to the people inside them. "Nothing makes sense except in light of inter-individual variation."5

Prevalence-Induced Concept Change

Here's a bias that might explain why progress feels invisible. Daniel Gilbert and colleagues demonstrated that as the prevalence of a problem decreases, people expand their definition of the problem to fill the gap.6 When shown fewer blue dots, participants started classifying purple dots as blue. When shown fewer threatening faces, they started classifying neutral faces as threatening. When reviewing research proposals on an ethics board and the prevalence of unethical studies decreased, they started flagging innocuous studies as unethical.

The effect persisted even when participants were warned about it and offered money to resist it. You literally cannot stop yourself from doing this by knowing about it — which is itself a lesson about the limits of debiasing through awareness.

The implications cut in two directions. On one hand, it explains why social progress tends to mask itself. As we reduce discrimination, we reclassify subtler behaviors as discriminatory. As crime drops, we redefine what counts as criminal. The world gets better but our standards ratchet up to match, producing the subjective experience of no improvement. On the other hand, sometimes expanding definitions is appropriate — an ER doctor should absolutely change her triage threshold when gunshot victims stop arriving. The problem is that we expand our definitions whether it's appropriate or not, and we can't easily tell the difference from the inside.6

This connects to Goodhart's law in reverse: instead of a measure being corrupted by optimization pressure, the measure itself drifts as conditions change, making it impossible to know when a problem has actually been solved.

The Meta-Bias

The deepest bias isn't any individual tendency — it's the belief that you can list your biases and thereby escape them. Knowing about biases can sometimes make you worse off, because it gives you a tool for dismissing other people's arguments ("that's just anchoring") without turning the same scrutiny on yourself. Munger knew this: his system wasn't a list of biases to memorize, it was a daily practice of intellectual self-discipline. Darwin's method of intensively considering disconfirming evidence wasn't a trick — it was a way of life.1

The strongest version of cognitive debiasing may not be individual at all. It may be institutional: designing organizations, incentive structures, and decision processes that route around the worst failure modes. FedEx didn't fix its workers' psychology — it fixed the incentive structure, and the psychology followed.

Footnotes

  1. Psychology of Human Misjudgment by Neil Kakkar (summary of Munger's speech) — source 2 3

  2. Are Sunk Costs Fallacies? by Gwern Branwen — source 2 3 4

  3. Book Review: The Secret Of Our Success by Scott Alexander — source 2 3 4

  4. Why Facts Don't Change Our Minds by Elizabeth Kolbert — source 2 3 4 5

  5. Different Worlds by Scott Alexander — source 2 3 4

  6. Prevalence-Induced Concept Change by Daniel Gilbert et al. (via ScienceDaily) — source 2

Open in stacked reader →