I obviously trust PPP. SUSA is good for the toplines, less good at crosstabs. Marist, CBS/NYT and ABC/WaPo are pretty solid. The internet pollsters—YouGov and Ipsos—were a curious (and ultimately successful) experiment. Pew is the gold standard, even when it's off. TIPP was a disaster in 2008, but it appeared more stable this time around. Some states have local pollsters so good they trump everything else, like Field in California and Selzer in Iowa. A few others were mildly interesting.
But there was a class of pollster that was so patently bad, they made me assume the whatever their results said, the opposite was actually true. So follow me below for a tour of this year's polling suck.
Steve Singiser's First Rule of Polling is, "If a poll doesn't look like the rest, it's likely wrong," and Gallup lived this mantra all cycle. While most polling showed a tight national race, Gallup consistently gave Romney 5-7-point leads.
Yet its long and storied history continued to give it credibility despite a disastrous recent track record. In 2010, Gallup claimed Republicans would win the Congressional national vote by 15 points. It was seven. In 2008, Gallup claimed President Barack Obama would win by 11. He won by seven. So how did Gallup save face? It used Hurricane Sandy as an excuse to quit polling for nearly a week, then delivered a late poll that showed Romney +1. No other pollster saw a major Romney erosion that week, and certainly not four points.
But even its last minute recalibration didn't save it, as its results put it 24 of 28 in accuracy. Below Rasmussen.
Quite the nice way to destroy their legacy.
These Republican hack pollsters single-handedly convinced Republicans, and some in the media, that Pennsylvania was a battleground state that Mitt Romney could win. At a time when the polling consensus was 7-8 points, they were claiming that Obama's lead was around two.
Their last poll this week had the race tied 47-47. Obama won by more than five.
Who can forget this highlight of the 2012 campaign?
“I think in places like North Carolina, Virginia and Florida, we’ve already painted those red, we’re not polling any of those states again,” [Suffolk University polling director David] Paleologos said Tuesday night on Fox’s "The O’Reilly Factor." “We’re focusing on the remaining states.”Funny thing was, Suffolk's own polling showed Obama in the lead! Yet he claimed that Obama was toast because undecideds would go to Romney. That kind of mistake might be understandable for those who haven't spent much time looking at polling data. Truth is, the 50 percent rule doesn't apply in presidential races. Someone who makes a living generating polling data should know better. As Armando wrote, Paleologos just ignored his own polling.
On 9/30, UNH had had Carol Shea-Porter up 46-35 in New Hampshire's 1st Congressional District. A week later, on 10/6, her opponent Frank Guinta was up 38-36. There was nothing in between to account for a one-week 13-point swing.
On election eve, 11/4, UNH had the race tied 43-43. Shea-Porter won comfortably by four. Such wild, unexplained swings (a hallmark of UNH results) are a mark of shoddy quality control.
Nate Silver ranked them the least accurate of 2010, and they'll likely earn the same this year:
In Colorado, Rasmussen polled at 50%-47% for Romney. The actual result was 51%-47% for Obama, the reverse of Rasmussen's poll.What's more, these final numbers were actually closer than some of their mid-year results, which were clearly designed to impact the polling aggregator numbers (and RCP, in particular) and to try and craft a "Romney is winning" narrative. This led, in a hilarious twist, to condemnation from the infamous "unskewing" guy:
In Florida, Rasmussen polled at 50%-48% for Romney. The actual result was 50%-49% for Obama, the reverse of Rasmussen's poll.
In Iowa, Rasmussen polled at 49%-48% for Romney. The actual result was 52%-47% for Obama, the reverse of Rasmussen's poll, doubled.
In New Hampshire, Rasmussen polled at 50%-48% for Romney. The actual result was 52%-47% for Obama, the reverse of Rasmussen's poll.
In Ohio, Rasmussen polled at a 49%-49% tie. The actual result was 50%-48% for Obama, a two-point swing.
In Virginia, Rasmussen polled at 50%-48% for Romney. The actual result was 50%-48% for Obama, the reverse of Rasmussen's poll.
In Wisconsin, Rasmussen polled at a 49%-49% tie. The actual result was 52-47% for Obama, a six-point swing.
[H]e said he probably won't go back to "unskewing" polls next time. He actually thinks conservative-leaning pollsters like Scott Rasmussen have a lot more explaining to do.MASON-DIXON
"He has lost a lot of credibility, as far as I'm concerned," [Dean] Chambers said. "He did a lot of surveys. A lot of those surveys were wrong."
M-D is a long-time respected member of the polling community. So what the hell happened to them in 2012?
They had Romney winning Florida 51-45. Obama won it by a point. They had the Republicans taking the Montana governorship 49-46. The Democrats took it by two.
They had Jim Matheson losing his congressional seat in Utah 50-43. He hung on by one. They had Republicans taking the North Dakota Senate seat by two. Democrats won it by one.
They had Claire McCaskill winning her Senate seat by two. She won it by 15. They had the Minnesota gay marriage ban pass by one point. It failed by almost 4.
In fact, it's hard to find any race of particular note that they got right.
No one knows where these jokers came from, but they spent October telling us how Romney was going to win Michigan. In fact, their election eve poll had it Romney 46.92-46.56. Any pollsters that reports results to a single decimal are suspect. Two? Pure wankery. Polling has inherent inaccuracies—hence the "margin of error". Pretending that results are so precise as to require multiple decimal points is simply inaccurate.
But aside from the decimals, their numbers were comical. Obama won Michigan by over eight points—a nine-point miss. They had Sen. Debbie Stabenow winning by just 50-43 (sorry, 50.06-43.45). Stabenow won 58-38. They even ventured into Florida to tell us that Romney led 54-40. They were laughed out of the state, never to return.
There were other crappy pollsters like Gravis, Zogby and ARG, and of course even the good ones had misses here or there. By definition, five out of every 100 polls will be off. But the pollsters above deserve every bit of scorn we can send their way, and then more.