The conventional wisdom is that New Hampshire was the most surprising event of the election cycle to date. Although New Hampshire was a true shocker, tonight's result in South Carolina actually qualifies that the bigger surprise from a statistical standpoint. Let's take a look at the actual voting tallies, as compared with the final polling averages from Real Clear Politics.
Polling Average Result Error
Iowa Obama +1.6 Obama +8.1 Obama +6.5
New Hampshire Obama +8.3 Clinton +2.6 Clinton +10.9
Michigan Clinton +23.5 Clinton +15.5 Uncommitted +8.0
Nevada Clinton +4.0 Clinton +5.5 Clinton +1.5
South Carolina Obama +11.6 Obama +28.9 Obama +17.3
The pollsters have not had such a good year so far. They underestimated Obama's margin over Hillary in Iowa by 6.5 points. Then more infamously, they overestimated his margin in New Hampshire by 10.9 points -- which resulted, among other things, in a Hillary Clinton victory. They also did poorly in Michigan, although that is a difficult race to poll.
However, tonight takes the cake. Obama's margin of victory in South Carolina was underestimated by a whopping 17.3 points -- about as large as the errors in New Hampshire and Iowa combined.
I've written before about how pre-election polls can be so wrong. The argument in a nutshell is as follows:
- Polls can't and don't account for the ultimate preferences of undecided voters, and it's not uncommon for undecided voters to make what amounts to a collective decision -- they break very heavily toward one or another candidate.
- Pollsters overstate the extent to which "decided" voters are actually decided; voting preferences in primary elections are extremely fluid.
In this case, undecided voters did in fact break as a group toward Barack Obama, and to a lesser extent toward John Edwards. Per CNN's exit poll, Obama won 52 percent of voters who decided in the last week, as compared to 27 percent for Edwards and just 20 percent for Clinton.
Why did undecided voters break away from Clinton -- and toward Obama and Edwards? There are a lot of parallels between South Carolina and New Hampshire (including the issue of "piling on"; this time Obama was the victim). In both cases, you had an extremely complex but extremely important dynamic in the narrative of the race that we didn't fully understand until after the fact.
In New Hampshire, that dynamic is hard to describe in a single sentence, but it can be described reasonably well with a few terms: Hillary "finding her voice", Obama overheating, the other candidates "piling on" Hillary. In South Carolina, that dynamic was Obama becoming combative and going toe-to-toe with the Clintons (and particularly Bill Clinton) on their negative campaigning.
As in New Hampshire, the punditry missed how this dynamic was playing out completely -- or at least with a small handful of exceptions like Josh Marshall, Joe Scarborough, and Mark Blumenthal (three people you should be looking up on a daily basis if you want good horse race coverage). As Blumenthal writes (citing Daily Kos diarist Eric Schmeltzer):
Clinton still holds enormous advantages on foreign policy, health care and the economy and for having the "right experience." "Honesty/integrity," however, is easily her her weakest dimension, and one of Obama's strongest, as other surveys have shown in recent months.
How important are perceptions of integrity and trust? Very. Drawing on decades of opinion poll data, political scientists identify two central traits -- competence and integrity -- that drive judgements about presidents and presidential candidates. "Presidents are judged," wrote Professor Donald Kinder (with whom I once studied at the University of Michigan), " by their intelligence, knowledge and experience on the one hand, and by their honesty, decency and ability to set a good moral example on the other" (p. 840). Candidates that are perceived to be otherwise qualified and competent lose when voters find them lacking in terms of honesty and trust. And keep in mind that the bulk of the research driving these conclusions comes from general election surveys in which perceptions of competence and integrity were sometimes strong enough to overcome partisan leanings in driving voter choices.
As Blumenthal notes, the Clintons lose on trust and integrity (from my point of view, with good reason). But until recently, it had not been a major focus of the campaign. The one time it had become important was after Hillary's "flip-flop" on drivers' licenses in the Philadelphia debate -- and that had been a key moment in shifting the momentum of the campaign. But otherwise we'd heard very little about it until the past 7-10 days.
The media focused on how Obama was playing a very aggressive form of defense -- but they neglected that there was an element of offense in his argument too:
"This is part of what we have to change. You know, folks willing to say anything just to get elected."
This is a potentially winning argument. Honesty/integrity is the tiebreaker between experience and change -- and if the election can be framed as a referendum on this issue, the Clintons will lose. But up until now, Obama had not been able to articulate this argument as well as he could. The voters of South Carolina figured it out -- but it took them until the last minute to do so. However, after Obama's victory speech tonight -- which is all about this argument, the rest of the country should have it a little easier.
I actually didn't get to watch too much of the post-election coverage -- we had a couple of friends over for dinner -- but from what I did gather, it was interesting to see the media narrative shift over the course of the night as the magnitude of what had happened sunk in. And certainly, Obama is in a good position to win most of the media cycles between now and Super Tuesday -- something which can help him significantly by allowing him to close positively while the media makes his critique of the Clintons for him.
However, let's return to the voters of South Carolina so that I can make my last point. Way back in November -- what now seems like ages ago -- I wrote a piece about the nature of momentum in the primary campaign, in this case in relation to Iowa. I ticked off several different varieties of "momentum" -- and concluded that one of the most important might be what I call Canary in the Coal Mine Theory:
[5] Canary in the Coal Mine Theory
This final theory breaks with the others in making few assumptions about voters at all. In fact, it need not assume the presence of such things as "momentum". Rather, it assumes that the Iowa Bounce is an artifact of the uncertain nature of polling. Coming up with likely voter models is notoriously difficult in primary elections, where only a small fraction of the state is likely to participate in the process, and even more so for state caucuses. Iowa, therefore, provides the first true indication of which candidates are strongest on the ground, and which voters are actually likely to turn out to vote in the election. Voters need not change their preferences after Iowa at all, but we may observe a momentum effect because Iowa tends to be a much better predictor of actual election outcomes in other states than polls are.
That is, irrespective of any actual "momentum" itself -- e.g. voters changing their votes because of what happened in an earlier state, what the results from the early states really provide is a leading indicator about how voters are eventually going to make up their minds. We've now seen what happens when voters have a chance to fully consider the honesty/integrity argument -- they gravitate away from Clinton and toward Obama and Edwards. And we're likely to see that effect now take hold at the national level. In fact, it may already be taking hold according to one prominent pollster.
Because of this, I suspect that South Carolina could have nearly as much impact on the race as did Iowa and New Hampshire -- which definitely produced bounces, as you can see below.
It isn't that South Carolina is all that important unto itself -- it received only some fraction of the media coverage of Iowa and New Hampshire. But it's the way that Obama won South Carolina that counts. He found a winning argument -- and one that's going to be much easier for him to articulate from here on out.