Often in error, rarely in doubt: confidence trumps expertise

June 14th, 2009

The new Scientist reports on a recent paper by CMU psychologist Don Moore that shows that people prefer advice from confident sources even when they have a poor track record.

Moore argues that in competitive situations, this can drive those offering advice to increasingly exaggerate how sure they are. And it spells bad news for scientists who try to be honest about gaps in their knowledge.

In Moore’s experiment, volunteers were given cash for correctly guessing the weight of people from their photographs. In each of the eight rounds of the study, the guessers bought advice from one of four other volunteers. The guessers could see in advance how confident each of these advisers was (see table), but not which weights they had opted for.

Describing his work at an Association for Psychological Science meeting in San Francisco last month, Moore said that following the advice of the most confident person often makes sense, as there is evidence that precision and expertise do tend to go hand in hand. For example, people give a narrower range of answers when asked about subjects with which they are more familiar”

Why aren’t we better at recognizing cover-confidence? There must be some evolutionary fitness in this, at least for humans. There can be a big penalty in indecision or vacillation. I wonder if we will see the same phenomenon in systems of cooperating autonomous agents?

Here’s the paper:

Joseph R. Radzevick and Don A. Moore, Competing To Be Certain (But Wrong): Social Pressure and Overprecision in Judgment, 21st Annual Convention of the Association for Psychological Science, May 2009.

Overprecision in judgment is both the most robust and the least understood form of overconfidence. Overly precise judgments claim more certainty than is objectively warranted. In this paper, we investigate whether the competitive social pressure of a market contributes to overprecision among those competing for influence. We find evidence that markets do indeed exacerbate overprecision. This evidence comes from two experiments in which advisors attempt to sell their advice. In the first experiment, advisors must compete with other advice sellers. In the second, advisors and decision makers are paired. Overprecision exists in both studies, and it helps advisors’ sell their advice. However, the market also exacerbates overprecision. We discuss the strategic implications of these results.

BlindSearch evaluates Google, Bing and Yahoo search engines

June 7th, 2009

Who’s got the best basic web search engine? One way to approach that question is to conduct an experiment in which subjects rank the results returned by several engines without knowing which is which.

BlindSearch is a simple and neat site that collects ‘objective’ opinions on search quality by showing query results from Google, Yahoo and Bing side by side without identifying which is which and inviting you to select the best.

“Type in a search query above, hit search then vote for the column which you believe best matches your query. The columns are randomised with every query.

The goal of this site is simple, we want to see what happens when you remove the branding from search engines. How differently will you perceive the results?”

BlindSearch evaluates Google, Bing and Yahoo

As of this writing there have been 1679 votes for preferred results with Google getting 39%, Bing 39% and Yahoo: 22%.

update 2:14pm edt 6/7: Google: 45%, Bing: 32%, Yahoo: 22% | 11,130 votes

Google Chrome for Linux and Mac

June 5th, 2009

How’s this for truth in advertising. The Chromium blog announces beta versions of Google Chrome for MAC OS X and Linus, but warns people not to try them in a post Danger: Mac and Linux builds available.

“In order to get more feedback from developers, we have early developer channel versions of Google Chrome for Mac OS X and Linux, but whatever you do, please DON’T DOWNLOAD THEM! Unless of course you are a developer or take great pleasure in incomplete, unpredictable, and potentially crashing software. How incomplete? So incomplete that, among other things, you won’t yet be able to view YouTube videos, change your privacy settings, set your default search provider, or even print.”

Of course, they know that this will make trying them irresistible to some of us. If that includes you, go get the Mac or Linux version.

Rising tide lifts all browsers

June 3rd, 2009

Mozilla’s Asa Dotzler posted some interesting graphs showing historical browser usage. Looking at the percentage of users, Internet Explorer is slowly losing market share to Firefox and Safari.

Looking at the total number of users, all three are increasing.

Bing vs. Google, side by side comparison

June 1st, 2009

Microsoft’s new Bing search engine is getting a lot of interest. Glenn McDonald posts about a nice side-by-side Bing vs Google comparator tat he developed. It makes it easy to compare how the two services do on a range of different types of searches. Here are the ones that Glen said he found useful in developing his initial opinion.

I sense form some of these queries that he is probing the systems where an advanced search engine can exploit a little bit of semantic knowledge. For example, recognizing that a user’s query “boston to asheville” matches a common pattern “ to “, and she probably is interested in information about how to travel from the first location tot he second. It seems like Google has been working on adding more such patterns, at least for the low hanging fruit.

Of course, if everyone hits on this site it may get throttled or blocked by either or both of the search engines. @Glen — would you be willing to share your code?

(spotted on hacker news)