UNH says there's been a 27-point swing against Republican Marilinda Garcia in only a week. That's hard to swallow.
There's one general truism it comes to the University of New Hampshire's polling: If you don't like the results, just wait—the next poll will show the opposite. UNH just released new polls of both of the Granite State's House districts, and for once, they defied the conventional wisdom by showing a stable race in the 1st District. Democratic Rep. Carol Shea-Porter leads Republican Frank Guinta 44-40, not too different from her 42-39 edge in their Oct. 7 poll.
But New Hampshire's 2nd District is another story. According to UNH, Democratic Rep. Annie Kuster is crushing Republican Marilinda Garcia 53-30, a huge reversal from Garcia's 41-37 lead in early October. Since that time, news broke that Garcia lifted other people's work verbatim in at least two speeches she delivered, without any attribution. And yes, she did generate some bad headlines. But enough to cause a 27-point swing in only three weeks? That does not sound remotely plausible.
By comparison, in the 2012 U.S. Senate race in Indiana, Republican Richard Mourdock's comments calling pregnancy from rape "an act of God" took him from what was probably a two-point lead (give or take) to a six-point loss. Yes, Indiana is far less swingy than New Hampshire and Democrats are capable of doing much better in the Granite State than in the Hoosier. But even so, Mourdock's comments were far worse than anything Garcia did, and even they produced "only" an 8-point swing to Democrat Joe Donnelly. In an era as polarized as this, it would probably take something along the lines of a murder conviction to cause a 27-point shift.
But there's more evidence that the Garcia kerfuffle was almost certainly not responsible for this huge gyration. As Daily Kos Elections' Dreaminonempty visualizes below, UNH has shown crazy swings in NH-02 for months, all without any scandal to blame:
UNH's 2014 NH-02 polling
UNH may be crazy, but Carol Shea-Porter is still in a tossup no matter what! Please chip in $3 to get her across the finish line.
Defeat Mitch McConnell in just two hours. Sign up to make GOTV calls to Democrats.
So what the Hell is going on here? Head below the fold to find out.
There's a much more obvious explanation for what happened here: UNH's tiny sample sizes make their House polls all but useless. In a recent post, Dreaminonempty vividly demonstrated how small sample sizes introduce a ton of noise into polls, creating swings where there are none. UNH's House polls routinely have sample sizes smaller than 300, which is unacceptably small. The simple solution is to poll more people, which would dramatically reduce the amount of turbulence in their polling.
But UNH's polling director Andy Smith has shown no interest in fixing the obvious problems with his polls. Indeed, UNH has used these tiny sample sizes for years. As far back as 2008, David Jarman lambasted UNH for this very issue:
Playing "spot the methodological weaknesses in the UNH poll" has become a favorite netroots pastime, and there are a few things to point out: each of the Congressional samples is only half of the statewide sample, leaving them with sample sizes of only 252 and 271, and enormous 6 percent MoEs.
Four years later, nothing had changed. In 2012, UNH also showed far too much bouncing in its House polling, featuring plenty more wild,
unexplained swings.
So why doesn't Smith care about the low quality of his surveys? It may be because he really has no incentive to fix the bugs. Media outlets, including Manchester's WMUR-TV and Maine's Portland Press-Herald, still keep paying Smith to conduct their polling. And the press will still breathlessly report his numbers no matter how awful they are. In fact, the more absurd the numbers, the more attention they get.
For instance, when UNH showed a 10-point swing in the New Hampshire Senate race in just one month over the summer, reporters breathlessly reported "Scott Brown's winning!" despite the firm's notorious history. The simple truth is that reporters like having interesting polls to write about, and they're quite happy to keep giving Smith attention no matter what.
UNH is far from the only pollster to ignore its own problems. As Fladem showed us in a 2012 essay, Gallup routinely produced poor surveys since at least 1996. The company had plenty of evidence that there was a major bug in their system and more than enough time to fix it, but they spent 16 years pretending all was fine.
And why wouldn't they: Reporters continually cited the Gallup as the gold standard for polling, even though they had done nothing to earn that honor for ages. It was only in 2012, when Gallup became a national punchline for blowing the presidential election, that they finally decided to admit that they had a problem. Unfortunately, UNH, with its more regional focus, is unlikely to ever receive this kind of scrutiny no matter how bad they are.
Smith deserves plenty of blame for producing poor polls and refusing to fix clear flaws. But ultimately, he really has no reason to change as long as both the media and the experts turn a blind eye to his methods. There is already so little accountability in the world of polling, and the fact that UNH's results are taken at all seriously by people who should know better only means we're going to see more terrible results from them—and from other lousy pollsters.
UNH may be crazy, but Carol Shea-Porter is still in a tossup no matter what! Please chip in $3 to get her across the finish line.
Defeat Mitch McConnell in just two hours. Sign up to make GOTV calls to Democrats.
NH-01 is a bellwether district and one we cannot afford to surrender. Please donate today to carry Carol Shea-Porter to victory on Tuesday.