Cross-posted here. This topic area has been touched upon at Daily Kos, both on the front-page and in some recent diaries, but I wanted to develop on it further.
For all of Politico's faults, they do produce some good stuff from time to time. This evening they published a really well-written and well-researched trend story on nail-biter election returns and the prevalence of court challenges. The apparent trend is even the foundation of the 2008 movie, "Swing Vote" that I saw last summer, where the whole presidential election comes down to one guy, who gets a do-over when the machine cancels his vote. The probability of that occurring is infinitesimal, but the likelihood of extremely close elections, according to the Politico story, is higher than ever before.
Here's the intro of the article:
As the Minnesota Senate recount drags into its sixth month and the March 31 New York special election looks likely to remain undecided for at least another week, now may be the time to ask: Why can’t voters pick a clear winner anymore?
At first glance, that might seem like an overstatement. After all, in 2008, Barack Obama won a convincing majority of the presidential vote and a preponderance of House and Senate elections were unambiguously decided.
Still, in a nation where the infamous Bush vs. Gore presidential stalemate of 2000 remains fresh in voters’ minds, recently there have been enough deadlocked contests in enough states to at least raise the question of whether federal elections are moving toward a frustrating new era of gridlock.
Initially, I was pretty skeptical that there was actually trend here, but as you can find in the piece itself, there were a surprising number of congressional races in 2008 that went past election night, and several states on the presidential level were close to tied. For some reason, it seems now that there are more close or court-challenged elections than in the past (excluding a few famous examples). So it's not that this is new, but people are noticing that the frequency is higher.
Andy Barr, who wrote the Politico piece, interviewed various campaign and political experts for historical background and possible theories for the trend. There are countless factors that could explain the trend, but I'll focus on one in particular here. The most interesting hypothesis (to me at least) suggests that micro-targeting of voters (as satirized in "Swing Vote") using modern technology has enabled candidates to get much closer to dead even than ever before. When once the victor was the candidate who could best appeal to a very broad segment of the electorate and could best campaign to swing large blocs, today it is possible for a weaker candidate to slowly campaign vote-by-vote toward a slim victory.
I'm currently working on a research project involving election trends from 1986-2008. I just finished data collection on state-by-state presidential results, and what I found is that in most of the elections, there was a very significant pool of electoral votes determined in their respective states by less than 5% margins, and in many cases by much less than 5%. (corrected for clarity)
The landslide election of Bill Clinton in 1992 featured a full 18 states that went for Clinton or Bush by less than 5%, accounting for 208 electoral votes split between them. So nationally, it looked like Bill Clinton had cleaned President Bush's clock, but in reality, he only achieved this by winning most of the close races state-by-state. That's a macro version of the vote-by-vote theory, and Gov. Clinton's team was and is often credited with having run a very modern and tech-savvy campaign. They focused on a wide range of states against Bush and won a lot of them by only a few points. But the electoral college means that this is all that really matters.
Sen. Barack Obama also did very well in both the primaries and the final election by employing a similar strategy. Spend a lot of money and time campaigning all over the place in order to pull off narrow wins in as many places as possible. Against Hillary Clinton, Obama piled up delegates by limiting her to narrow wins in many states (meaning he was apportioned nearly the same number as she was even while losing a state) and by concentrating on his best areas and districts to crush her delegate count slowly. It took a while, but it worked. In the fall, he forced McCain to spread limited resources thinly by opening up new states that had been safe for Republicans in 2004. I predicted for Indiana and North Carolina, for example, that Obama would win narrowly. He did. Those two states put 26 electoral votes in the bank that Dems hadn't been able to get in 2000 or 2004, when each race was decided by less than that. So Obama may have gotten 95 more electoral votes than necessary, but most of those were a few points away from going to McCain instead. Obama did not have to try to win those 95 votes by swinging the whole country, just as Bush did not win the 2000 race overall, but won by capturing Florida with an official margin of under 300 votes. As quite a few observers suggested throughout the campaign, Obama was doing so well in the electoral college, that McCain could have won the popular vote and still lost the presidency.
It's seems like a risky strategy to go for the super-narrow win, but it's actually easier and more efficient for many candidates. If you were starting at a disadvantage, you would especially be more likely to try employing a highly targeted campaign to get just enough to cross the finish line ahead. The targeting is now highly sophisticated and heavily researched. When I volunteered for the Obama campaign, a New Hampshire-based operative told me that the research showed that the best way to win was to keep going back to the same voters over and over, in person, via phone, etc. If it took driving out to pick up a voter to get them to the polls on election day, that would happen. Had Al Gore swung 7,211 voters in New Hampshire in 2000, he would have won the presidency. Gov. Bush was able to concentrate on those voters, in a state where under 600,000 voted, and that prevented Gore from winning it all. Sure it was closer in Florida, but New Hampshire had fewer total voters and could be more easily micro-targeted. That's why the Obama Campaign was taking it so seriously voter-by-voter, though it ended up being a solid Democratic victory there in 2008.
There is an excellent site (warning, very long page) that calculates the minimum number of votes that a presidential candidate would have needed to swing in individual states, in order to win the election. I've checked back through the second half of the twentieth-century and it seems clear that it's much closer these days than ever before. John McCain could have won in 2008, by swinging (a total of) 445,912 votes in seven specific states. On the other hand, Barry Goldwater in 1964 would have needed to convince 2,058,258 Johnson voters in 26 specific states in order to win. That's an extreme comparison, but it does suggest that elections today are (partially) closer because of improved targeting that closes the gap in more states, without worrying as much about the national picture. McCain was much closer than Goldwater to pulling off a win because he ran far closer in more states (...and just because he wasn't Goldwater running against Johnson). Obama happened to run slightly farther ahead in those states, and for that I am glad.
The scattershot/hit-or-miss campaigns of the past may be over, though bad candidates and poor campaigns will persist. There will still be some landslide wins, but they may become less frequent. Again, this isn't the only reason for the trend, but it is a big factor.