As Election Day approaches, the volume of polling has increased. It’s easy for any individual poll to be wrong for a variety of reasons. However, we can reduce the chance for error from any single poll by combining it with many other polls in a statistical average, which is just what Huffington Post Pollster and Daily Kos Elections do.
These averages are essential for giving us the lay of the land in key races, like those that could determine which party controls the Senate in 2016. Of course, averages are only as accurate as the underlying body of polls they collect, and even these aggregates can miss a race by several points or fail to predict the correct winner. Polling in 2012 systematically underestimated Democrats while 2014 polls did the opposite. Still, some states have polling errors more consistently in favor of one party than other states do.
Below, we’ll take a look at Pollster’s averages over the last four elections compared to the actual results in the four states we currently rate as “Tossups” in the fight for the Senate: Indiana, Nevada, New Hampshire, and Pennsylvania. Although the low number of races for each state unavoidably yields a small sample size that lacks strong predictive power, we can still investigate whether these polling errors might still be able to tell us anything about this year’s Senate battlegrounds.
Can you contribute $3 to each of the six Senate candidates endorsed by Daily Kos?
Polling errors in Indiana seem to be the most consistent of this quartet of states. In all three cycles shown in the chart above, the polls slightly underestimated the Democratic margin by an average of about 2 percent. However, unlike the other three states, Indiana had no race for Senate or governor in 2014, when polls generally overestimated Democrats nationwide. Still, it’s possible that 2016 could continue the trend of recent election cycles.
However, most pollsters tend to shy away from Indiana because of a state law prohibiting automated-caller surveys, and live-caller polls cost much more to conduct. That leaves us with an even less robust average that relies heavily on a smaller number of polls. Coupled with only a modest skew against Democrats in recent years, we shouldn’t expect that Democrat Evan Bayh, who has led in all polls to date against Republican Todd Young, will overperform his party’s polling again in 2016 based on this data alone.
Nevada is perhaps the chief case where analysts regularly assume—rightly or wrongly—that polls have a systematic bias against Democrats. The polling average infamously predicted Senate Majority Leader Harry Reid’s demise in 2010, yet he ultimately prevailed, yielding a particularly large polling error that year. And although 2014 saw an even bigger error in the opposite direction, we nonetheless have better reason for concluding there is consistent bias against Democrats than even in Indiana.
First, Nevada has one of the nation’s highest proportions of residents who recently moved into the state. Second, it has a very high share of urban households which only have cellphones. Finally, it has a sizable Latino population that might be less likely to answer surveys in English instead of Spanish. These three factors can make it much more difficult for pollsters to get a representative sample of the electorate, at least if they aren’t willing to pay for more expensive methods to reach them. Indeed, pollsters who offered a Spanish option tended to be more accurate in recent cycles.
Finally, there’s the case of 2014. Polls didn’t just underestimate the Republican margin, they did so massively. However, that year’s gubernatorial election was never competitive like races in all other years, so pollsters rarely ran surveys there. Compounding matters, Democrats didn’t even bother fielding a credible candidate, and because their voters weren’t sufficiently mobilized, turnout was among the nation’s lowest.
That made it even more difficult for pollsters to determine who was likely to vote. But this year, with a competitive presidential election driving high turnout in Nevada, pollsters should have an easier time figuring out who will actually vote. If we were to discount the uncompetitive 2014 race, the average polling discrepancy favored Democrats by 5 percent on average instead of 0.7 percent.
That’s a very big margin compared to the other states, enough so that a poll showing a close race could very well call the wrong winner. Indeed, with Republican Joe Heck leading Democrat Catherine Cortez Masto in the race for Harry Reid’s open seat by just a modest 43-40 average, an error in Democrats’ favor could make a very big difference.
New Hampshire, which conducts gubernatorial elections every two years, has seen its polling errors jump all over the place in the last four cycles. The year 2012 underestimated Democrats while 2010 skewed against Republicans, but 2008 and 2014 had fairly modest errors. New Hampshire is a relatively homogenous state with a less-transient population compared to Nevada, so there’s little underlying demographic reason why we should expect any large polling errors here. But even a small one could have a profound impact, since Democrat Maggie Hassan and GOP Sen. Kelly Ayotte are currently tied at 46 apiece in the HuffPo average.
Unlike the other three states, Pennsylvania’s handful of polls have underplayed Democratic chances in presidential years, when turnout is high, yet painted an overly optimistic picture for the party in midterms, when turnout is much lower. Like Nevada, it’s possible there could be an underlying demographic reason causing this discrepancy, such as if pollsters have trouble gauging black voter turnout, but we have less evidence to support that hypothesis than in the Silver State.
These races ultimately balance out to a relatively trivial impact on average without accounting for midterms versus presidential years. But if this presidential-versus-midterm pattern plays out again, that could redound to the benefit of Democrat Katie McGinty, who right now has a 44-42 edge on Republican Sen. Pat Toomey.
Aside from Nevada, where there have been consistent signs of polling errors underestimating Democrats, there’s simply not enough information to predict such non-trivial polling misfires in the other three top-tier battleground Senate races. With polling generally more volatile in this election than in 2012 or 2008, it’s entirely possible we could have big errors come Election Day. However, recent results simply can’t predict much about 2016 errors except for this: While often more accurate than any individual poll, even averages can be wrong.
Let's take back the Senate. Please chip in $3 to each of our six Daily Kos endorsed Senate candidates.
Frustrated you don't live in a swing state? No matter where you live, MoveOn has a great way for you to help their on-the-ground efforts to defeat Donald Trump and take back the Senate. Click here to volunteer.
