What's the connection between a 1906 poultry exhibition and the 2008 US election?
The Betfair Prof discusses the wisdom of crowds and wonders about the best way of reading the polls.
Sir Francis Galton was an English explorer, anthropologist, scientist, who was born in 1822 and died in 1911. To students of prediction markets he is best known, however, for his visit, at the age of 85, to the West of England Fat Stock and Poultry Exhibition, and what happened when he came across a competition in which visitors could, for sixpence, guess the weight of an ox.
Those who guessed closest would receive prizes. About 800 people entered. Ever the scientist, he decided to examine the ledger of entries to see how clever these ordinary folk actually were in estimating the correct weight. In letters to 'Nature' magazine, published in March of 1907, he explained just how ordinary those entering the competition were. "Many non-experts competed", he wrote, "like those clerks and others who have no knowledge of horses, but who bet on races, guided by newspapers, friends, and their own fancies ... The average was probably as well fitted for making a just estimate of the dressed weight of the ox as an average voter is of judging the merits of most political issues".
The results surprised him. For what he found was that the crowd had guessed (taking the mean, i.e. adding up the guesses and dividing by the number of entrants) that the ox would weigh 1,197 pounds. In fact, it weighed 1,198 pounds! The median estimate (listing the guesses from the highest to the lowest and taking the mid-point) was also close (1,207 pounds, and therefore still within 1% of the correct weight) but not as close. Some have argued that Galton himself favoured the use of the median rather than the mean, and so was double-surprised when the mean beat the median. Others have argued that the point is incidental and what this tale demonstrates about the wisdom of the crowd is more important than such a fine statistical detail.
I think that both these points of view contain some merit. The power of the market to aggregate information is indeed a critically important idea. But it is also important to be able to distinguish in different contexts which measure of the 'average' (the mean, the median, or perhaps some other measure) is more suited to the purpose at hand.
Take the stream of opinion polls which contribute to the collective knowledge that drives the Betfair market about the identity of the next President of the United States. If five are released, say, on a given day, what is the most appropriate way of gauging the information contained in them? Should we simply add up the polling numbers for each candidate and divide by the number of polls, or should we list them from highest polling score to lowest and take the mid-point. The convention adopted by sites such as www.realclearpolitics.com is to take the mean. But is there a better measure than the mean of discerning the collective wisdom contained in the polls, and if so, what is it? The jury is still deliberating.'.$sign_up['title'].''; } } ?>