The Betfair Prof: The election day polls performed almost perfectly - on average!

The Betfair Prof / Leighton Vaughan Williams / 10 December 2008 / Leave a Comment

The Betfair Prof, Leighton Vaughan Williams, takes a look at how the polls performed on election day back on 4th November for the US Presidential Election...

And so now we know. The votes cast in the 2008 US Presidential election have all been counted and tallied and we know for sure what the markets and the models and the polls could only hint at. Senator Barack Obama of Illinios earned 52.9% of the popular vote, Senator John McCain took 45.7% of the vote, and the remaining 1.4% was split between assorted third-party candidates.

So how did the final polls published by the respective opinion polling organisations perform? RealClearPolitics published most of them, and displayed them on its website on election day. These ranged in sample size from just 714 likely voters (CBS News) to 3,000 (Rasmussen Reports), and covered survey dates ranging from as early as 29th October to 1st November (Pew Research) at one end to 3rd November only at the other (Marist).

If we take the polls containing the biggest sample sizes (more than 2,000 voters) we obtain the following:

- Rasmussen Reports (3,000 likely voters): Obama 52%; McCain 46%
- Pew Research (2,587 likely voters): Obama 52%; McCain 46%
- Gallup (2,472 likely voters): Obama 55%; McCain 44%
- ABC News/ Washington Post (2,470 likely voters): Obama 53%; McCain 44%

This gives a big-sample average as follows: Obama 53%: McCain 45%. This is an 8% margin in favour of Obama compared to the actual margin of 7.2%.

If we now take the polls with the latest survey dates we obtain the following:

- Marist (3rd November): Obama 52%; McCain 43%
- Battleground (Lake projection - 2/3 November): Obama 52%; McCain 47%
- Battleground (Tarrance projection - 2/3 November): Obama 50%; McCain 48%

Actually this is one poll divided into two according to different methodologies, so should be counted only once (an average of the methodologies gives: Obama 51%; McCain 47.5%).

This gives a late-survey average as follows: Obama 51.5%; McCain 45.3%, a 6.2% margin in favour of Obama compared to the actual margin of 7.2%

So we have something of an over-estimate in one case and an under-estimate in the other.

There are a total of 14 polls (counting the alternative Battleground methodologies as one poll), ranging from highs of 55% (Gallup) and 54% (Reuters/C-SPAN/Zogby) for Obama to lows of 50% (Fox News) and 51% (NBC News/Wall Street Journal). For McCain the polls ranged from a high of 47.5% (Battleground average) to a low of 42% (CBS News).

So what happens if we simply take all the final polls published on RealClearPolitics and divide by the number of polls? This is without regard to the dates of the surveys or the sample sizes or the methodology of the poll but simply taking a bare average of everything on offer.

Well, we obtain the following: Obama 52.1% (actual: 52.9%); McCain 44.5% (actual: 45.7%).

This represents an advantage of 7.6% for Obama (taking an unweighted, unadjusted average of all these polls), 0.4% off the actual margin in favour of Obama of 7.2%. Now add in the final Daily Kos/Research 2000 poll which RealClearPoltics for their own reasons decided to exclude from any of their daily summaries and what do we obtain? An Obama spread of 7.4%, within 0.2% of the final tally!

And so there we have it! The election day polls performed almost perfectly - on average!

Professor Leighton Vaughan Williams is the Director of the Political Forecasting Unit and Betting Research Unit of Nottingham Business School, Nottingham Trent University