Chapter 21: Intuitions vs. Formulas

Type: Chapter 21

Category: Psychology

Difficulty: Intermediate

Reading time: 6 min

Core idea

In 1954, Paul Meehl published a study that should have changed how clinical professionals make predictions: in almost every domain where a simple statistical formula competed with expert clinical judgment, the formula won. Meehl reviewed studies across psychiatric diagnosis, parole decisions, medical prognosis, academic performance prediction, and credit risk. The formula — a simple weighted sum of a few variables — outperformed or equaled the clinician’s holistic judgment in nearly every comparison.

In the fifty years since Meehl’s study, the evidence has only accumulated. More than 200 studies across domains confirm the finding. Yet clinical practice, hiring decisions, investment management, and strategic planning continue to rely on expert intuition rather than actuarial models. The resistance to formulas is itself a psychological phenomenon worth understanding.

Why it matters

Why formulas beat experts in low-feedback domains

Formulas have several advantages over human judgment in predictive tasks:

• Consistency: a formula gives the same answer for the same inputs every time. Human experts are inconsistent — the same case presented on a Monday vs. a Friday receives different ratings. Inconsistency is uncorrelated noise that reduces predictive accuracy.
• No fatigue or distraction: expert judgment degrades over time, across a long day of decisions, and when extraneous information is present. Formulas do not.
• No confirmation bias: formulas weight evidence by its statistical association with the outcome, not by its vividness or narrative fit.
• No overweighting of extreme information: experts tend to over-adjust for dramatic or memorable cases in a way that formulas do not.

The expert does not add information — they add noise. In domains without fast, accurate feedback, the noise compounds rather than the signal.

Even simple formulas are hard to beat

More surprising: even simple, equally-weighted formulas — that just add up the relevant variables — often outperform experts. Experts disagree about how to weight factors, and their disagreement introduces variance. An equal-weighting scheme eliminates disagreement and produces a consistent aggregate. The consistency advantage alone is enough to beat expert consensus in many domains.

Why the resistance persists

Kahneman acknowledges why this evidence is resisted: it is deeply uncomfortable. It implies that the expertise people have developed over careers — the ability to read people, to sense nuance, to integrate complex signals holistically — adds noise rather than signal in many prediction tasks. It threatens professional identity.

More subtly, the resistance is generated by cases where experts are obviously right — when they catch something a formula would miss. These cases are memorable and vivid; the cases where the formula would have been right and the expert was wrong are diffuse and invisible. The availability-affect system defends expert judgment against the statistical evidence.

Author’s argument: The argument is not that experts never know anything. It is that in any specific prediction task, you can test whether expert judgment adds value over a formula by comparing predictions to outcomes. Until that test is done, the claim of expert superiority is not evidence — it is an illusion of validity.

Key takeaways

Key takeaways

• Formulas vs. clinicians: simple actuarial models consistently outperform or equal expert clinical judgment across 200+ studies in low-feedback prediction domains.
• Why formulas win: consistency — formulas give the same answer for the same inputs every time. Expert judgment is noisy, fatigues, and varies by context.
• Even equal-weighting models beat experts: the main advantage of formulas is not superior weighting but elimination of inconsistency and disagreement.
• The resistance is predictable: expert judgment feels right because memorable cases where experts catch what formulas miss dominate over diffuse cases where formulas win.
• The test that is rarely done: compare prediction accuracy systematically. Until outcomes are tracked against predictions, claims of expert superiority are unchecked illusions of validity.
• Implication: use formulas for prediction where possible; reserve expert judgment for tasks where formulas genuinely cannot capture the relevant variables.

Mental model

click to zoom

Why formulas beat expert judgment in low-feedback domains — consistency vs. noise.

Read it as: Given the same inputs, a formula produces the same output every time. An expert produces variable output depending on context, fatigue, and framing. Across many cases, the formula’s consistency means its errors are uncorrelated noise; the expert’s inconsistency adds correlated noise that reduces predictive validity. The formula wins not because it is smarter, but because it is more consistent.

Practical application

Where to use formulas

Formulas work best for prediction tasks where: (1) the outcome is measurable; (2) there is a historical dataset of inputs and outcomes; (3) the prediction environment is relatively stable; (4) feedback is delayed or noisy enough that experts cannot learn reliably from experience. Hiring, loan approval, clinical diagnosis, and recidivism prediction all satisfy these conditions.

Implementation guidance:

• Build the formula first: identify the five to ten most predictive variables from research or historical data. Assign weights based on correlations with outcomes, or use equal weighting. Apply it consistently.
• If using expert judgment, standardize the inputs: ask the expert to rate specific dimensions independently before forming an overall judgment. This reduces the mental shotgun contamination and gives the expert’s holistic judgment a better chance of competing with the formula.
• Track predictions against outcomes: the only way to test whether expert judgment adds value over a formula is to run the comparison. If you never track, you will always believe in the expert.

Example

A law school admits students using a combination of LSAT scores, undergraduate GPA, and an admissions committee interview. The committee interviews 800 applicants and rates each on “potential.” When first-year law school grades are compared to predictions, LSAT + GPA alone predicts grades better than LSAT + GPA + committee ratings.

The committee’s holistic judgment not only failed to add predictive value — it slightly reduced accuracy. The committee is introducing inconsistency and narrative-based evaluations that introduce noise into a signal that already exists in the quantitative data. The solution is not to improve the committee’s judgment but to build a formula from the quantitative predictors and stop using the committee for prediction purposes — reserving it for evaluation tasks that cannot be formalized.

Related lessons

Related

Chapter 20: The Illusion of Validity The confidence that experts feel in their predictions — which exists regardless of whether those predictions are accurate. Chapter 22: Expert Intuition: When Can We Trust It? The conditions under which expert intuitions are valid — the complement to this chapter's critique. Dual-Process Theory System 1 and System 2 — the psychological framework that explains why intuitive judgment is inconsistent and why formulas capture the consistent signal.