The Betfair Prof / Leighton Vaughan Williams / 18 August 2009 / Leave a comment

The prediction market master looks at the effects of 'optimism bias' with the help of Google...

"Optimism bias was found to be more evident in new employees and in the immediate few days following a good news day for the Google stock price. Still, what is a cost in terms of unadjusted predictive efficiency may be a benefit in terms of motivation and entrepreneurial zeal, a feedback mechanism the value of which it is perhaps easy to under-estimate."

Optimism bias, according to the latest Wikipedia entry, is "the demonstrated systematic tendency for people to be over-optimistic about the outcome of planned actions. This includes over-estimating the likelihood of positive events and under-estimating the likelihood of negative events".

To call it a "demonstrated systematic tendency" is perhaps pre-calling the result of the debate about the nature and existence of such a bias, but you get the general idea. Indeed, David Armor and Shelley Taylor highlight a number of examples of what they consider to be optimism bias in an interesting paper called 'When Predictions Fail: The Dilemma of Unrealistic Optimism', published in 2002.

Examples include students' estimates of the likely starting salary of their first job in the graduate market and newlyweds' thoughts on how long their marriage will last. It is interesting, therefore, that evidence of the existence of this very same bias has been identified in 'internal' company prediction markets, notably in a 2008 paper co-authored by Bo Cowgill, of Google, Justin Wolfers of the Wharton School and Eric Zitzewitz, based at Dartmouth College.

The distinguished authors of the study examine the results generated by what they call the Google corporate prediction market experiment. The primary goal of these markets is, as they put it, to generate predictions that efficiently aggregate many employees' information and augment existing forecasting methods.

In support of previous investigations into the value of internal prediction markets, they were able to confirm that prices in the Google markets closely approximated event probabilities, i.e. that the markets were reasonably efficient. Even so, they were not perfect, and one notable reason was an apparent 'optimism bias' which, according to their findings, "was more pronounced for subjects under the control of Google employees, such as whether a project would be completed on time or whether a particular office would be opened."

Optimism bias was also found to be more evident in new employees and in the immediate few days following a good news day for the Google stock price. Still, what is a cost in terms of unadjusted predictive efficiency may be a benefit in terms of motivation and entrepreneurial zeal, a feedback mechanism the value of which it is perhaps easy to under-estimate.

In any case, if we are able to identify and measure the source and extent of the bias, it should be possible (in whole or part) to adjust and compensate for this particular inefficiency in generating the objective forecasts.

So is 'optimism bias' a particular issue for internal prediction markets? The Google paper identifies the bias as particularly evident where the subject of evaluation was to some extent under the control of the players. This is less evident in the case, for example, of political prediction markets. Each individual player is likely to contribute only a tiny fraction to the total outcome.

So can we identify an optimism bias in these macro-markets and if so what impact is it likely to have, and can we adjust for it? Are Republicans, for example, more optimistic than Democrats, and how might that affect the forecasts of US elections? And how can we distinguish over-optimistic trading of one candidate from straight market manipulation? These are important questions. The challenge now is to help provide the important answers.

Tags: Google, Optimism Bias, Prediction Markets