Often in error, rarely in doubt: confidence trumps expertise

June 14th, 2009

The new Scientist reports on a recent paper by CMU psychologist Don Moore that shows that people prefer advice from confident sources even when they have a poor track record.

Moore argues that in competitive situations, this can drive those offering advice to increasingly exaggerate how sure they are. And it spells bad news for scientists who try to be honest about gaps in their knowledge.

In Moore’s experiment, volunteers were given cash for correctly guessing the weight of people from their photographs. In each of the eight rounds of the study, the guessers bought advice from one of four other volunteers. The guessers could see in advance how confident each of these advisers was (see table), but not which weights they had opted for.

Describing his work at an Association for Psychological Science meeting in San Francisco last month, Moore said that following the advice of the most confident person often makes sense, as there is evidence that precision and expertise do tend to go hand in hand. For example, people give a narrower range of answers when asked about subjects with which they are more familiar”

Why aren’t we better at recognizing cover-confidence? There must be some evolutionary fitness in this, at least for humans. There can be a big penalty in indecision or vacillation. I wonder if we will see the same phenomenon in systems of cooperating autonomous agents?

Here’s the paper:

Joseph R. Radzevick and Don A. Moore, Competing To Be Certain (But Wrong): Social Pressure and Overprecision in Judgment, 21st Annual Convention of the Association for Psychological Science, May 2009.

Overprecision in judgment is both the most robust and the least understood form of overconfidence. Overly precise judgments claim more certainty than is objectively warranted. In this paper, we investigate whether the competitive social pressure of a market contributes to overprecision among those competing for influence. We find evidence that markets do indeed exacerbate overprecision. This evidence comes from two experiments in which advisors attempt to sell their advice. In the first experiment, advisors must compete with other advice sellers. In the second, advisors and decision makers are paired. Overprecision exists in both studies, and it helps advisors’ sell their advice. However, the market also exacerbates overprecision. We discuss the strategic implications of these results.