Previously, we saw that today's Senate polls, on average, will have an error of about seven points in the margin, if they are anything like the polls in 2010 and 2012.
What about polling averages? Many, including myself, advise not to worry about individual polls and follow the polling averages instead. How well did they do?
Below you can see the comparison of polling averages computed 50 days from the election of 40 Senate races in 2010 and 2012. In this case and the next graph, I am using the value of the local regression of the poll margins at a given day instead of a simple average.
In this graph, we would like to see the points sitting on the diagonal line, or close to it, meaning the polling average was correct.
Update: points above the line show where Democrats did better than the polling average, and points below the line show where Republicans did better than the polling average.
Instead, on average, they're 5.4 points away from it. That means that polling averages, today, are about 5 points off from the actual result, in one direction or another, if 2014 is similar to 2012 and 2010.
Only three races had polling averages 50 days away from the election that were within one point of the actual margin. Seven races were more than 10 points away.
Fine, so that's 50 days away from the election. What about right before the election?
Find out below the fold.
Here's the graph by the final polling averages, calculated one day before the election:
The error has been reduced, and there are fewer points wandering about far from the line. The average error is now 4.0 points, and the largest error is 8.8 points. Nine of the polling averages have an error of less than one point.
However, it's also clear that the errors are not random. In fact, this pattern is an old friend showing up—a portion of the error is due to the partisan lean of the state, as can be seen in this graph.
So far, so good. Senate polling averages have had a fair amount of error 50 days out in 2010 and 2012, that's reduced and partially predictable by election day. If 2014 is like 2010 and 2012, then polling averages should be very useful indeed.
But—2014 may not be like 2010 and 2012: there are, so far, more races with third-party candidates garnering more than a few percent. And the problem with that is that races with third-party candidates receiving more than 5 percent of the vote have more, and unpredictable, error. Additionally, the mix of pollsters this year is different too, with more Republican-affiliated pollsters of dubious quality.
So what's to be done? Well ... follow the polling averages. But also keep a wary eye on third-party candidates, and rogue Republican pollsters.