Prediction Market Theory: What an 18th Century English Reverend Can Teach 21st Century Juries!

The Betfair Prof / Leighton Vaughan Williams / 09 March 2009 / Leave a Comment

How reliable is DNA evidence in solving crimes? Not as reliable as you might think, says the Betfair Prof, Leighton Vaughan Williams...

A trace of DNA is found at the scene of the burglary. The scientists come up with a random match probability between the DNA found at the scene and the DNA of the defendant of one in two million. Now, the only witness to the crime described the burglar as in his mid-20s. A suspect was arrested and an identity parade was arranged. The witness failed to pick him out, and agreed that the suspect didn't fit the description he had given of the burglar. In particular, he agreed that the suspect looked about 40.

Moreover, the suspect had an alibi for the night in question: his wife was adamant that he had spent the night at home and was with her watching TV at the time the burglary occurred. So the DNA was the only incriminating evidence heard by the jury. Even so, one in two million is a very small number.

Now, what is your forecast of the outcome of the jury deliberations? If your guess is anything like mine, you will predict the jury will return in short order with a unanimous pronouncement of 'guilty'. After all, juries love DNA evidence. But would they be applying the correct analysis in reaching their verdict?

Well, what does a DNA 'match probability' (as it is termed) of one in two million really mean? Does it mean that the chance of anyone else being the culprit is one in two million? If so, that would be damning. In fact, it means that in Great Britain (population about 60 million) there will be on average about 30 people whose DNA matches that found at the crime scene. There may be more, there may be less. Now the task of the jury, if they perform it properly, is to decide on the basis of the other evidence, whether or not they are sure beyond a reasonable doubt that it is the defendant who is guilty, rather than one of the 30 or so people with matching DNA.

This is where a theorem proposed originally by the Reverend Thomas Bayes comes in. A well-informed jury might decide to implement the theorem by producing a questionnaire which asks a series of questions such as:

a. If he were the burglar, what's the probability that the witness would say that the suspect looked nothing like the burglar?

b. If he wasn't the burglar, what's the probability that the witness would say he looked nothing like the burglar?

These probabilities can now be entered into a formula derived from Bayes' theorem, first published in 1763, to produce an overall assessment of the probability of guilt. The formula itself can be written as: P (AIB) = P (BIA) P (A) / P (B).

In words, this can be translated as: The probability of A given B equals the probability of B given A times the probability of A, divided by the probability of B. That's the easy bit. The hard bit is putting numbers into the formula, especially good numbers. This doesn't mean we shouldn't try.

And was he guilty? Well, the jury had never heard of Bayes' theorem and convicted unanimously. And did he do it? Ask his 25-year old brother!

Professor Leighton Vaughan Williams is the Director of the Political Forecasting Unit and Betting Research Unit of Nottingham Business School, Nottingham Trent University