The Skeptics Society & Skeptic magazine

How Doctors Think They Think

a book review by Charles Lambdin

When an expert writes a book for the public, he is writing for an audience largely unqualified to assess its claims, an audience primarily composed of uncritical consumers. This creates a problem: How can those genuinely curious but untrained sift accurate explanations of science’s best thinking from anecdotally-derived hoopla? Unfortunately, Jerome Groopman’s How Doctors Think is not always clear on this problem.

Groopman is at his best discussing the deleterious impact of HMOs, the influence of big pharma, and especially the “medicalization” of aging. He turns a skeptical eye to “erectile dysfunction,” “male menopause,” and even “ADHD,” arguing that pharmaceutical companies sometimes create drugs for nonexistent illnesses, and then produce a market for their product by inventing a “disease” for the drug to treat. Are older men’s erectile “dysfunction” and plummeting testosterone levels “illnesses,” or normal parts of aging that have been medicalized? If a child is bored and restless in the fourth grade, should he be prescribed Ritalin or advanced to the sixth grade? When asking such questions, Groopman reminds me of Thomas Szasz, who defiantly believed that mental illnesses are invented, not discovered.

Much else in the book, however, fails to impress. One would hope that a book entitled How Doctors Think would be about just that, how doctors make diagnoses, a topic on which much scientific research has been done. Unfortunately, Groopman by-and-large ignores this research, instead giving us a long collection of anecdotes wherein his doctor friends tell us how they think. And this is the book’s main failing, for if you were to write a book purporting to examine how some group of people thinks, you would have to do more than simply ask some of them.

Groopman tells us that many diagnoses are wrong, and that most medical errors are the result of cognitive biases resulting from quick and dirty rule-of-thumb heuristics we all use to make decisions when pressed for time, operating under uncertainty, or when we simply do not know a better decision rule. He tells us that doctors are often unaware of the impact such heuristics — and their accompanying biases — have on their diagnostic accuracy.

This is an excellent point, but it begs a simple question: What if the same cognitive biases that wreak havoc on doctors’ thinking — unawares — also influences their thinking about their thinking?¹ In How Doctors Think, doctors tell us how simply spending more time with patients, asking open-ended rather than close-ended questions, reading “body language,” including patients’ emotions in the picture, etc., improves their diagnostic ability. How much impact do these factors have on diagnostic ability? Do they actually have any impact at all? These questions place Groopman’s book on shaky ground, for his evidence consists primarily of unrepresentative and mostly irrelevant testimonials in which doctors describe various atypical diagnostic adventures they’ve had, and how they think they either emerged victorious or dropped the ball.² The problem is, this does not tell us anything about how doctors think, only how they think they think.

The problems get worse. Groopman tells us he is troubled that new doctors seem to be trained to “think like computers,” that they rely on diagnostic decision aids and some seductive “boiler-plate scheme” called evidence-based medicine. Groopman’s position, when his various arguments are gathered and assembled, becomes untenable. He admits doctors suffer from innumerable biases that diminish the accuracy of diagnosis, reducing many diagnoses to idiosyncratic responses fueled by mood, whether the patient is liked or disliked, advertisements recently seen, etc. Thus Groopman agrees with decision scientists’ diagnosis of doctor decision making; but then he goes on to wantonly dismiss what many of the very same researchers claim is the best (and perhaps only) remedy, the way to “debias” diagnosis: evidence-based medicine and the use of decision aids. In place of statistics what does Groopman suggest doctors rely on? Clinical intuition of course, the very source of the cognitive biases he pays lip service to throughout his book.

Unfortunately, what research Groopman cites to back his claims is somewhat one-sided and sometimes off the point. Two articles he cites both argue that decision aids pertaining to treatment (rather than diagnosis) don’t take into account when patients have multiple illnesses requiring multiple medications, which may interact with each other.^3,4 This is an important point, but to attempt to argue from this single issue that decision aids shouldn’t be relied on is to make a rather specious generalization.

Most doctors do not like decision aids. They rob them of much of their power and prestige. Why go through medical school and accrue a six-figure debt if you’re simply going to use a computer to make diagnoses? One study famously showed that a successful predictive instrument for acute ischemic heart disease (which reduced the false positive rate from 71% to 0) was, after its use in randomized trials, all but discarded by doctors (only 2.8% of the sample continued to use it).⁵ It is no secret many doctors despise evidence-based medicine. It is impersonal “cookbook medicine.” It is “dehumanizing,” treating people like statistics. Patients do not like it either. They think less of doctors’ abilities who rely on such aids.⁶

The problem is that it is usually in patients’ best interest to be treated like a “statistic.” Doctors cannot outperform mechanical diagnoses because their own diagnoses are inconsistent. An algorithm guarantees the same input results in the same output, and whether one likes this or not, this maximizes accuracy. If the exact same information results in variable and individual output, error will increase. However, the psychological baggage associated with the use of statistics in medicine (doctors’ pride and patients’ insistence on “certainty”) makes this a difficult issue to overcome.

The statistics vs. clinical intuition debate has ensued for decades in psychology.⁷ Where one sides in the debate is largely determined by what one makes of a single phrase: “Group statistics don’t apply to individuals.” This claim, widely believed, ignores many of the most basic concepts of probability and statistics, such as error. Yes, individuals possess unique qualities, but they also share many features that allow for predictive power.⁸ If 95% of a sample with quality X has quality Y, insisting that someone with quality X may not have Y because “statistics don’t apply to individuals” will only decrease accuracy. Insistence on certainty decreases accuracy. As Groopman himself says, the perfect is the enemy of the good.

Groopman, quite bizarrely, then stresses the uncertainty inherent in every diagnosis while maintaining that “statistics don’t apply to individuals.” Uncertainty is exactly why statistics apply to individuals: they give us the best guess available given the level of uncertainty at hand. It does not behoove anyone to ignore uncertainty and pretend it is not there. When doctors hear a decision aid has a 25% error rate and state “I can beat that,” this should raise suspicion. This is sometimes called the “broken leg argument.”⁸ Imagine an algorithm predicts when people will go to the movies with 80% accuracy. I think I can beat that if I happen to know a particular individual has a broken leg, but what if I start treating every diagnosis as a “broken-leg problem?” Will my accuracy increase, or decrease?

Physicians who allow themselves to think in such discretionary ways can find “exceptions” everywhere they look, and, augmenting a decision aid as they see fit, will only end up lowering its overall diagnostic accuracy. Why? Because human beings do not apply rules consistently. Mechanical procedures always lead to the same conclusion from the same input. Doctors are subject to random fluctuations in diagnosis caused by judgmentally-irrelevant factors including availability, priming, recency effects, inconsistent weighting of information, fatigue, etc., all of which reduce accuracy. What leads to a correct decision for one case may not for another, and variables that contribute to the diagnosis made may actually be uncorrelated with it.^6,7,8

Groopman makes comparisons to the perspicacious (and fictitious) “deductions” of Sherlock Holmes. Such comparisons are disingenuous. Most medical patients are not unique mysteries with atypical solutions (though the anecdotes in Groopman’s book are). Accuracy under uncertainty often boils down to what types of errors one is willing to make: a few false negatives with exotic conditions, or a great many “zebra-chasing” false positives. Oddly, Groopman rebukes doctors guilty of “zebra retreat,” but bungles the example this term is derived from. He quotes: “When you hear hoofbeats, think about horses, not zebras.” The actual lesson is, “When in Wyoming, if you hear hoofbeats and think you see stripes, it’s still probably a horse.” Groopman scoffs at this, which is unfortunate because treating every patient as a uniquely rare case will only increase error over time. When one looks at the research Groopman largely ignores, the lesson is clear. Medicine needs to become more science and less art.

References & Notes

1. Wilson, T. & Schooler, J. 1991. “Thinking Too Much: Introspection Can Reduce the Quality of Preferences and Decisions.” Journal of Personality and Social Psychology, Vol. 60 (2), 181–192.
2. For a discussion on why we often cannot accurately gauge the accuracy of our own judgments see: Einhorn, H. & Hogarth, R. 1978. “Confidence in judgment: Persistence of the illusion of validity.” Psychological Review, 85, 395–416.
3. Tinetti, M. 2004. “Potential pitfalls of disease-specific guidelines for patients with multiple conditions.” New England Journal of Medicine, 351, 2870–2874.
4. O’Connor, P. 2005. “Adding Value to Evidence-Based Clinical Guidelines.” Journal of the American Medical Association, 294, 741–743.
5. Corey, G. & Merenstein, J. 1987. “Applying the Acute Ischemic Heart Disease Predictive Instrument.” Journal of Family Practitioners, 25, 127–133.
6. Arkes, H., Shaffer, V. & Medow, M. 2007. “Patients Derogate Physicians Who use a Computer-Assisted Diagnostic Aid.” Medical Decision Making, 27(2), 189–202.
7. Grove, W., Zald, D., Lebow, B., Snitz, B. & Nelson, C. 2000. “Clinical versus Mechanical Prediction: A Meta-Analysis.” Psychological Assessment, 12(1), 19–30.
8. Dawes, R., Faust, D. & Meehl, P. 1989. “Clinical versus Actuarial Judgment.” Science, 243, 1668–1674.