The 2008 Presidential election is over. Now it's time to grade the pollsters who raised our hopes, confounded our thinking, made us reach for antacid tablets, and more often than not set the narrative for the media's coverage of the race. First up...Mason-Dixon, the pollster I personally love to hate because of their GOP house lean, which Nate Silver has documented. M-D has a contract with MSNBC and Chuck Todd is required to pimp them every time they do a poll dump. Their house lean (and their gross underestimation of the actual Democratic percentage vote) often forms the basis of MSNBC/NBC's political narrative for the day, which tends to upset the stomachs of many Democrats.
Mason-Dixon has an outsized reputation. In 2004 they had an average to above average year. In 2006, they had a bad year, getting 1 key Senate race wrong (RI) and underestimating Democratic support in a number of Senate races. Let's see how they did this year.
1. Virginia: Actual Result: Obama 52 McCain 47. M-D Final Poll: Obama 47 McCain 44.
Mason-Dixon's final poll was essentially a push: Obama 47 McCain 44. While they got the winner right and were only 2 points off on the margin, they left so many undecided on the table that the poll gave us little guidance as to how the race would turn out. Their write-up suggested that McCain would make the late gains because most of the undecided were white voters.
As it turns out, Obama got 52% of the vote, a number that was predicted by other polls. By M-D's own standards, Obama won the undecided vote, raising his percentage 5 points while McCain picked up 3. Therefore, if undecided voters were mostly white, they seemed to break evenly or slightly more for Obama than McCain. Though this race was a nail-biter to watch as the vote count unfolded, it turned out to be a fairly comfortable win for Obama, in line with most public polling, and a remarkable +13 turnaround from the 2004 Bush-Kerry finish. M-D had a house lean in this race and one could look to other polls to get a better idea of who was going to win. Obama had been over 50% in most of the polls taken since October 1. He finished with 52% of the vote.
2. Ohio: Actual (current) Result: Obama 51 McCain 47. M-D Final Poll (10/29) McCain 47 Obama 45.
M-D totally blew Ohio. Obama scored an impressive 7 point turnaround from Kerry's 2004 performance. He hit 51%, a number predicted by other pollsters (and a lot of votes still left to count in Dayton and one other blue county). M-D severely underestimated Obama's number, and got McCain's number right. However, their analysis did not indicate Obama's true strength in the state and was a classic outlier. [There are still votes to count in Dayton and one other blue county in NE OH, so Obama's margin will likely increase at least slightly towards a 5 point margin].
3. Florida: Actual Result: Obama 51 McCain 49. M-D Final Poll: Obama 47 McCain 45
M-D's result was more in line with other polls and got the winner and margin correct. However, they continued to leave a lot of undecided on the table, giving us little guidance as to how the race would turn out. Other polls were more instructive.
4. North Carolina: Actual Result: Obama 50 McCain 49. M-D Final Poll: McCain 49 Obama 46
Plain and simple, M-D got NC wrong. While they accurately predicted McCain's end result, they underestimated Obama by 4 points, a significant miss given how close everyone believed the race was in the state. There were several other polls that were much closer to the actual result. PPP probably had a better read on NC than any other pollster as they got this one just about right.
5. Colorado: Actual (current) Result: Obama 53 McCain 46. M-D Final Poll: Obama 49 McCain 44
There are still a lot of votes to be counted in Colorado, and Obama is likely to increase his margin by around a half a point to a point as Boulder and a few other blue counties still have votes outstanding.
Based on the current result, M-D missed the margin by 2 points and as usual, underestimated Obama's number by at least 4 points. Other polls consistently had Obama at over 50% and Colorado ended up being one of the easiest calls for the media organizations to make on election night. This race was not close, but M-D's numbers only suggest a relatively small lead, not a dominant one.
6. Nevada: Actual Result: Obama 55 McCain 43 - M-D final poll: Obama 47 McCain 43
Nevada was one of Obama's best states on election day. The double digit margin was a surprise to many, but a few did see the race going this way (Suffolk and Zogby). M-D's numbers were essentially useless as an instructive guide to the race as other pollsters gave a better indication of Obama's strength (many showing him at or above 50%). As usual, M-D did not push leaners and made it appear that the race was more fluid than it was in reality.
7. Pennsylvania: Actual Result: Obama 55 McCain 44. M-D final poll: Obama 47 McCain 43.
This is another bad poll by M-D. Every pollster had Obama ahead in the state, so the real value is margin, undecided, and top line result. M-D missed all 3. They underestimated Obama's top line number by a whopping 8 points, understated the margin by 7 points, and had many more undecided voters than other polls. There is a huge difference between a 4 point margin and an 11 point margin. If the race really had been that close, Obama would have spent a lot of time and money there, and probably may have sacrificed other states like Indiana, North Carolina and possibly Florida. It's a good thing the Obama campaign didn't use M-D as its pollster.
8. Missouri: Actual Result: McCain 50 Obama 49. M-D final poll: McCain 47 Obama 46
It looks like M-D made the right call on this one (though I am still hopeful for a recount in MO, as the margin is less than 5,000 votes). M-D refuses to push leaners, and in this case it may have been justified given the tightness of the race. Everything in MO boiled down to turnout, and a number of conservative Democrats voted for Jay Nixon for governor but then turned around and voted for McCain for President. That erosion of conservative Democratic support to McCain offset the increase in black turnout and made it possible for McCain to seemingly eek out a narrow victory. Missouri is still infected by the red bug, and the fundie movement there has crystallized to tilt the state red. It is no longer a true bellweather as it has now sided with the bad guys in the last 3 elections and went against the national popular vote twice (2000/2008). Furthermore, in 2004, the popular vote margin in MO exceeded the actual national percentage by about 5 points. MO is a far cry from the state that voted twice for Bill Clinton.
[as an aside, M-D had not done a poll of New Mexico since September, but they had McCain up 5 points when no other poll showed such strength. Obama won the state by double digits].
Conclusion
In short, Mason Dixon did not have a very good night on Election Night '08, and I am so glad for that. It was hard enough to watch the stomach churning vote count in Virginia (which ended up in line with most public polling). I would have hated for the other key races to have been close.
Mason-Dixon's methodology needs a tune up for a brand new electorate. We are not a Republican country anymore (and never were), as it was the Democratic Party, not the GOP, that won the close ones this time. The Obama campaign was built to win close elections and that confidence comes from the belief that there are more potential Democrats than Republicans in many states that have not previously been receptive to national Democratic candidates. M-D was often the outlier in key states and they got 2 key races completely wrong: Ohio and North Carolina. M-D made assumptions about undecided white voters that did not hold true in a number of cases. Had whites reverted to 2000 and 2004 voting patterns, Obama would not have won Ohio, North Carolina, Florida or Indiana.
Had the Obama campaign relied on M-D as its pollster, they would have sacrificed about 80 electoral votes focusing on states that they didn't need to while leaving real pickup opportunities on the field. Even in the states they got 'right' there were many other state polls that gave a clearer picture of the race. So, for getting 2 races wrong and underestimating Obama's share of the vote in just about every key state, M-D gets a grade of D.