Skip to main content

Scott Brown in his pickup, giving a thumbs up.
Scott Brown may be giving a thumbs-up to this latest poll. Sensible analysts aren't.
Like the small state it's in, the University of New Hampshire receives outsize attention when it comes to politics. Thanks to their regular polling of a state that often hosts contested elections (not to mention its first-in-the-nation presidential primary), UNH has ensured that political observers will always talk about the data it churns out, justified or not.

And it's not justified. UNH is one of the worst pollsters out there.

That didn't stop traditional media reporters from breathlessly declaring that New Hampshire's Senate race, which Democratic Sen. Jeanne Shaheen has had well in hand for some time, was suddenly a "dead heat" when UNH released a new survey Thursday evening showing her up just 46-44 on Republican Scott Brown. (Remember him? The former Massachusetts senator who seems to have a hard time remembering he's carpetbagging to a different state?) And boy, check out UNH's trendlines!

Trendline of UNH's 2014 NH-Sen polling between Jeanne Shaheen (D) and Scott Brown (R)
The problem, though, is that no other reputable pollster has shown the race that close all year. In fact, here's what things look like without UNH in the mix:
Aggregate chart of 2014 NH-Sen polling between Jeanne Shaheen (D) and Scott Brown (R), not including UNH
Now, UNH's numbers are a bit fresher than what's out there, so it's worth asking whether anything has dramatically changed in this race since last month, when UNH found Shaheen up by a 50-38 margin. But Andy Smith, who runs UNH's polling center, barely even tried to offer an explanation for this huge shift, claiming that Shaheen's campaign was being "being weighed down by national politics, particularly the declining popularity of President Obama." Say what?

Please read below the fold for more on this story.

Aggregate chart of Barack Obama's job approval ratings for 2014
Those train tracks are a chart of Obama's national job approval ratings from all pollsters from the start of 2014 until today. The guy's not popular, clearly, but if one thing's for sure, his popularity hasn't declined—and certainly not since July alone. Did Andy Smith even look at this data before pronouncing Shaheen a victim of "Obama drag"? It's hard to imagine that he did.

But that's standard operating procedure at UNH, where inexplicable gyrations are forever ignored, brushed aside, and swept under the White Mountains. How could you possibly explain this impossible zig-zag in another race UNH has polled often, New Hampshire's 1st Congressional District?

Trendline of UNH's 2014 NH-01 polling between Carol Shea-Porter (D) and Frank Guinta (R)
Are we honestly meant to believe that Democratic Rep. Carol Shea-Porter was up 16 points on Republican Frank Guinta in October, only to find herself down 6 in January, then back up by 9 in April, and then back down by 3 in July? New Hampshire politics may be volatile, but this is cockamamie stuff. Smith didn't even try to justify this, merely claiming that "[s]upport for Shea-Porter has remained stable" (not true) while "support for the lesser-known Guinta has fluctuated somewhat." Fluctuated somewhat!

(And as an aside, please note that we called out UNH earlier this year when Shea-Porter was riding high. Our objections to UNH have nothing to do with whether their polling is good for Democrats or bad for Democrats. UNH's suckiness is a serious problem no matter which party they seem to be favoring.)

One thing UNH's defenders like to point out is that the school's last polls of the cycle have often been closer to the mark, and that's true. For instance, they found Obama up 5 points in New Hampshire just before Election Day 2012; he won by 6. They also had Democrat Maggie Hassan winning the governor's race by 9; she prevailed by 12. But that hardly exonerates them. UNH's presidential polling was absolutely berserk over the final month of the race:

Trendline of UNH's 2012 presidential polling in New Hampshire between Barack Obama (D) and Mitt Romney (R)
Again, we're meant to believe that Obama started October up 15 points, then collapsed into an outright tie a month later before surging to a 5-point edge in the final few days before the election? No friggin' way—and we know this just didn't happen because the Obama campaign's own national polling, the most accurate in the land, never budged more than a couple of points all year long. So who you gonna trust, David Axelrod or Andy Smith?

All that said, it's certainly possible that New Hampshire's Senate contest has tightened. But even if for some reason you trusted UNH completely, you should never rely on one single poll to form an opinion on a race. Certainly you'd never try to predict the outcome of a baseball game based solely on the score after one inning. It may well be that the next poll—the next 10 polls, even—all show the same thing as UNH has here. If that happens, then maybe UNH was right. It's certainly not impossible, after all. But it's also possible they'll just get lucky—or will be entirely wrong. And given how erratic they are, the latter is a safer bet.

So why does UNH produce such erratic polling? No one can truly say. There are lots of bad pollsters out there, whether in Republican boiler rooms or on leafy college campuses, though it probably doesn't help when your interviewers are bored undergrads trying to get through call lists as quickly as they can to earn school credit. Still, there should be much greater quality control at the top. No respectable outfit would release numbers than bounce around so comically without first trying to get to the bottom of the problem.

But that, of course, means recognizing you've got a problem in the first place. And when you're used to getting lots of attention, it must be awfully tempting to look the other way when you've got a headline-grabbing poll like UNH's latest. Beltway reporters still have a lot of affection for Scott Brown, whom they imagine actually lives up to the barn coat-wearing, regular-guy-Republican image he's always tried hard to project. And they also love a good horserace. Andy Smith gave 'em one, whether it exists or not.

Originally posted to Daily Kos Elections on Fri Aug 22, 2014 at 10:09 AM PDT.

Also republished by Take New Hampshire Forward! and Daily Kos.

Your Email has been sent.
You must add at least one tag to this diary before publishing it.

Add keywords that describe this diary. Separate multiple keywords with commas.
Tagging tips - Search For Tags - Browse For Tags


More Tagging tips:

A tag is a way to search for this diary. If someone is searching for "Barack Obama," is this a diary they'd be trying to find?

Use a person's full name, without any title. Senator Obama may become President Obama, and Michelle Obama might run for office.

If your diary covers an election or elected official, use election tags, which are generally the state abbreviation followed by the office. CA-01 is the first district House seat. CA-Sen covers both senate races. NY-GOV covers the New York governor's race.

Tags do not compound: that is, "education reform" is a completely different tag from "education". A tag like "reform" alone is probably not meaningful.

Consider if one or more of these tags fits your diary: Civil Rights, Community, Congress, Culture, Economy, Education, Elections, Energy, Environment, Health Care, International, Labor, Law, Media, Meta, National Security, Science, Transportation, or White House. If your diary is specific to a state, consider adding the state (California, Texas, etc). Keep in mind, though, that there are many wonderful and important diaries that don't fit in any of these tags. Don't worry if yours doesn't.

You can add a private note to this diary when hotlisting it:
Are you sure you want to remove this diary from your hotlist?
Are you sure you want to remove your recommendation? You can only recommend a diary once, so you will not be able to re-recommend it afterwards.
Rescue this diary, and add a note:
Are you sure you want to remove this diary from Rescue?
Choose where to republish this diary. The diary will be added to the queue for that group. Publish it from the queue to make it appear.

You must be a member of a group to use this feature.

Add a quick update to your diary without changing the diary itself:
Are you sure you want to remove this diary?
(The diary will be removed from the site and returned to your drafts for further editing.)
(The diary will be removed.)
Are you sure you want to save these changes to the published diary?

Comment Preferences

Subscribe or Donate to support Daily Kos.

Click here for the mobile view of the site