The Betfair Prof: 'Could the crowd have prevented 9/11?'

The Betfair Prof / Leighton Vaughan Williams / 26 October 2009 / Leave a Comment

The Japanese attack on Pearl Harbor in December, 1941, came as a surprise to the US intelligence community. The attack by al Qaeda on the World Trade Centre and on the Pentagon in September 2001, came as a similar surprise. But the information was out there.

So why the surprise? Simply put, it was because the systems were not in place with which to put the jigsaw together in time. The 9/11 Commission Report stated the problem like this: "The biggest impediment to all-source analysis - to a greater likelihood of connecting the dots - is the human or systemic resistance to sharing information."

James Surowiecki, in his book, 'The Wisdom of Crowds', offers the following perspective on this failure: "What was missing in the intelligence community ... was any real means of aggregating not just information but also judgements. In other words, there was no mechanism to tap into the collective wisdom of National Security nerds, CIA spooks, and FBI agents. There was decentralization but not aggregation ..."

The question is whether the market can help achieve this. Some people within the US Department of Defence already thought so, and had been working on just such an idea for several months when al Qaeda struck. Indeed, in May 2001 the Defense Advanced Research Projects Agency (DARPA) had issued a call for proposals under the heading of 'Electronics Market-Based Decision Support' (later 'Future Markets Applied to Prediction' (FutureMAP).

The remit prescribed for FutureMAP was to create market-based techniques for avoiding surprise and predicting future events. It was not long, however, before the media and key members of the political class, began to train their guns on the idea of such a market. After all, it isn't difficult to portray what was devised as a policy analysis market as no more than a forum for eager traders to profit from death and destruction. The populist arguments won the day and DARPA was forced to cancel the project.
While most of the arguments against the market were specious, there was some genuine intellectual concern as to how effective it would be likely to be.

In particular, Joseph Stiglitz, winner of the 2001 Nobel Prize for economics, argued in an article published in the Los Angeles Times on 31 July 2003 ('Terrorism: There's No Futures in It'), that the market would be too "thin" (i.e. there would be too little money traded in the market) for it to be a useful tool for predicting events meaningfully. His argument was based on work he had previously published showing that markets can never be perfectly efficient when information is costly to obtain. The cost of obtaining and processing this information is, by implication, likely to act as a significant disincentive particularly in the context of a thin market (and hence low rewards).

I am not so sure. Is it obviously the case that a properly constructed market, populated by suitably motivated (and perhaps screened) players can be viewed in this way?
Well, the jury's still out on this, and in the meantime so are the so-called 'terrorism futures'. For how long, I wonder?