One of the main problems with running a model, especially one that's polls-only, is that your final result is only as good as the polls that you feed into it. You can do everything right in terms of averaging out the polls and running simulations, and still get tripped up because the polls themselves weren't getting a representative-enough sample of the population, or were making incorrect assumptions about who's in the pool of likely voters. The general idea behind poll aggregation is that if you get enough polls, you can smooth out each individual poll's particular errors and more accurately describe what's really happening. What happens, though, when
every poll in a particular state is suspect?
We try to be skeptical of every poll, but there are some states that especially set our antennae twitching. States that are heavily white, and stagnant in terms of population growth, tend to be easier to poll correctly than states with significant non-white or non-English-speaking populations, or states where there are a lot of people moving in or out. One of the states that falls in the latter category is Colorado, which has a substantial Hispanic minority and also has a lot of population churn as it rapidly grows; it's also been the site of an effective Democratic ground game in recent elections.
In the Colorado Senate race, we've seen Republican challenger Cory Gardner putting up a small but consistent lead in polls for the last month. Part of the reason we're skeptical of that is because recent history hasn't really borne that out. In 2010's Senate race, the majority of polls put Republican Ken Buck in the lead (he lost narrowly to Michael Bennet), while in the 2012 presidential race, a number of polls put Mitt Romney in the lead, even though Barack Obama went on to win the state by more than 5 points. The most notorious example was a Quinnipiac poll that put Romney up by 5, but on the whole, 2012 polling in Colorado had a noticeably higher average error than polling in other swing states.
And now we've had several developments in the last few days that further call the reliability of Colorado polling into question. One was that SurveyUSA, one of the more reliable pollsters and also one that works to incorporate new technologies, issued two different polls of Colorado from the same timeframe, one on behalf of the Denver Post and the other on behalf of High Point University. What's most interesting about this is that the two polls used different methods; both were a mix of robo-dialers and online contacts, but the Post poll used random digit dialing and the High Point poll used registration-based sampling (which relies on voter files).
Registration-based sampling is often considered a better approach, since it lets you zoom in on people likelier to vote instead of casting a very broad net. However, the random digit poll captured a larger, and seemingly more accurate Hispanic population; it had a 16 percent Hispanic component (in line with the 2012 exit polls, where Latinos were 14 percent of the sample), while the RBS poll only was 6 percent Hispanic.
We'll continue picking apart these polls, and look at changes in the model, over the fold:
Part of the problem is that voter registration in Colorado doesn't include race (because it isn't subject to Voting Rights Act preclearance), so the RBS poll's sample couldn't be weighted to race. Another potential problem, though, is the RBS poll's rigorous likely voter screen, which would require a voter to have voted in both 2010 and 2012 or to have registered since then; that could exclude a lot of voters who might not vote with regularity but who will be voting this year (thanks to nagging from Democratic GOTV efforts ... and, more importantly, thanks to the fact that this will be an all
mail-in election for the first time, which makes voting less of a hassle and boosts turnout, if turnout in Oregon and Washington elections is any indication).
There wasn't a huge difference between the results of the two polls; the random digit dialed poll found Gardner leading by 2, while the registration-based sample poll found Gardner leading by 4 (which is similar to other findings from RBS pollsters, like CNN's Wednesday poll, which also put Gardner up 4). However, the RBS poll also found an unusually Republican-friendly Hispanic subsample, with Mark Udall leading only 52-31 among Hispanic voters.
(Poll junkies are already familiar with this common quirk to SurveyUSA's crosstabs: Hispanic and black voters, as well as young voters, often end up unusually Republican-friendly. SurveyUSA's Jay Leve acknowledged as much, without offering much explanation, in a fascinatingly candid interview with the New York Times' Nate Cohn. The interview is a must read, if you're interested in how the polling sausage gets made.)
Conveniently, another Colorado poll came out on Tuesday as well, which further casts some suspicions on the RBS poll. It's a poll of only Hispanic voters in Colorado, conducted for the NCLR Action Fund by the pollster Latino Decisions, who specialize in polling Hispanics. The poll was a mix of cellphones, landlines, and Internet, and, unlike most other pollsters, was conducted in either English or Spanish by bilingual interviewers. It found that 14 percent of the electorate is Hispanic (suggesting the random-dialed poll got that right), but also that the Hispanic electorate is breaking much more heavily for Udall than SurveyUSA found: 66-17, with leaners pushed. It also found less of an enthusiasm gap than you'd think, with 43 percent saying that they're more enthused to vote in 2014 than they were in 2012 (compared with 34 percent saying they were more enthused in 2012).
Put this all together, and we're not very convinced that the bulk of the pollsters looking at Colorado are getting this race right. If the race weren't especially competitive, none of this would matter, but if it's the difference between, say, a 1-point Gardner win and a 1-point Udall win, it makes a major difference in calculating the Colorado odds. And with Colorado one of the linchpin races (along with Iowa and Alaska) that could determine Senate control, you can see how that uncertainty just gets amplified when calculating the overall Democratic odds!
The only alternative, though, to just throwing all these polls on the pile and hoping for the best, would be to start adding various "unskewing" procedures of downweighting or recalculating poll numbers. That just introduces more sources of error without giving us any more certainty, so we're just going to forge ahead with our Senate numbers.
The two SurveyUSA polls plus the CNN poll collectively push Udall's odds of victory in Colorado down further, to 28 percent (from 35 percent on Monday). The Democrats also had some erosion in Iowa (where Suffolk, Rasmussen, and Quinnipiac polls all gave Joni Ernst very small leads), where Bruce Braley's odds fell from 42 percent to 33 percent ... and in New Hampshire, where Jeanne Shaheen's odds are still fairly healthy but the lowest they've been, at 69 percent, down from 82 previously. (That's thanks to a 1-point Scott Brown lead in a poll from New England College; they, like UNH, tend to bounce around randomly, and I wouldn't be surprised to see Shaheen leading by 7 in next week's NEC poll.)
Together, that's enough to push overall Democratic Senate odds from 34 percent down to 29 percent. The only Dem candidate to keep gaining is, oddly enough, Allison Lundergan Grimes in Kentucky, whose odds are now up to 35 percent from 26 percent on Monday, thanks to a 3-point deficit in a Gravis poll (which, because of their Republican pollster status, pencils out to a tie for our purposes). That gives her better odds than four different Dem incumbents (not just Udall, but also Mark Begich, Mark Pryor, and Mary Landrieu) ... and yet she just got triaged by the DSCC, so while it's likely that Dem internals don't look as good for her as the public polls do, it's also possible that the decision was more based on that she's fundraised so well that she has enough of a financial cushion to get turned loose.
In what's become a consistent theme lately, the Dems' gubernatorial odds continue to go in the opposite direction, hitting a new high with a 65 percent chance of gaining seats, up from a 63 percent chance on Monday. Big gainers include Maine, up to 61 percent Dem odds from 52 percent on Monday (with Mike Michaud holding a 6-point lead in a new Ipsos poll), and Hawaii, where David Ige has moved into pretty safe-looking territory (from 84 percent odds to 92 percent odds today) after posting a 10-point lead in an internal poll from GSG.
Perhaps the best polling news on Wednesday came from the gold-standard Marquette Law poll of the Wisconsin gubernatorial race, showing a tie between Mary Burke and Scott Walker, up from a 5-point deficit several weeks ago. That wasn't that different from other polls, though, and it didn't quite get her head above water, boosting Burke's odds only from 43 to 46 percent today. Also, there's one alarming gubernatorial number, as Sam Brownback continues to work his way back into the Kansas race where he'd seemed DOA earlier; Dem challenger Paul Davis's odds have fallen below water for the first time, down to 46 percent from 56 percent earlier, after posting a tie in a PPP poll and a 3-point deficit in a poll from GOP pollster Remington.