Yesterday we talked some about what can cause polling errors. But let's cut to the chase: we really want to know how well today's polls can predict election outcomes in November.
This has an easy answer, for Senate polls at least: on average, the margin of a poll you see today will be wrong by about 7 points, in one direction or another, assuming 2014 is similar to 2012 and 2010. That means some polls will have a lot less error, and some will have a lot more—maybe even 20 points off.
The graph below shows the errors of all the Senate polls from 2012:
The y-axis shows the difference (absolute value) between the poll's margin and the result margin.
You can see there were plenty of polls, even less than ten days before the election, that missed the final margin by 10 points or more. The line is a local regression.
But what about 2010? Peek below the fold to see.
2010 was a very different year, as far as the electoral environment goes. Were the trends in the errors any different for Senate polls?
No, not until you get way back to early 2009 polling and the after-glow of the 2008 election. This gives us a little confidence that the errors in 2014 could behave in the same way.
While we're at it, here's a graph of the errors of the 2014 primary polls:
It looks pretty similar, until you notice the y-axis. Here's the local regression from all three graphs plotted together:
Clearly, primary polls, as a whole, have more error than Senate general election polls. We also see that on average, Senate polls taken within ten days of the election have a margin that is five points off in one direction or another. A couple months away from the election, and that rises to seven points.
Another way to look at it is to plot the final election margins versus the poll margins. Here's the plot for 2010 Senate polls completed in the final 10 days of the campaign:
Perfect polls fall right along the diagonal line. Compare to polls taken 11-30 days out, and those taken 31-60 days away from the election:
There's quite a lot of error there, and it's not evenly distributed. (We'll talk more about that tomorrow).
With such uncertainty, what's a poor pollwatcher to do? Average them, of course. That's tomorrow's post.