It’s happening again. My wife has caught me stomping around the house, muttering that someone is wrong on the internet, dealing with ghosts of the past. Exit poll bunk has returned, and I am... grumpy.
Just in the last week or so, a young man named Spencer Gundert and a less young man named Tim Robbins (remember him?) have written similar articles intended to draw people’s attention to what they regard as evidence of election fraud. Both writers make generalized observations about the security vulnerabilities of voting machines. Both writers present lists of things that have gone wrong during the primaries — several of which are real problems, although none has been established as willful fraud. (If you’re a voter stuck in line and/or forced to cast a provisional ballot because of erroneous registration lists and other fiascoes, you may not care much whether you and your neighbors are experiencing malice, incompetence, or some intermediate condition.)
But these lists document specific problems in specific places — say, Brooklyn, not the entire state of New York. As far as we know, the vast majority of Democratic (and Republican) primary voters have uneventfully cast their votes. To pitch a master narrative of fraud, one needs something more. Here the exit polls come in. Long story short: in most of the recent primaries, at least, Hillary Clinton has done better in the vote count than in the exit polls, often substantially.
These exit poll discrepancies evidently helped convince Gundert that “this Democratic nomination was rigged.” Robbins asks plaintively, “if we cannot rely on exit polls, then what is to prevent people from hacking into electronic voting machines to change people’s votes?” (Try replacing “exit polls” with “love, sweet love” or “chicken entrails” in that sentence. It’s a counsel of despair masquerading as an appeal to... reason? I can’t even.)
Many decent people have been led to believe in a strong or weak form of what I call exit poll fundamentalism: basically, the notion that exit polls in their “original” form are highly accurate reflections of voter intent, and any discrepancies probably evince error or fraud. They’re understandably miffed when this belief is dismissed out of hand as “conspiracy theory.” (But it does, in fact, entail a sort of conspiracy among the large number of expert political observers who unaccountably fail to speak out about this supposedly damning evidence.) What are they getting wrong? Here’s my quick take:
1. The myth of accuracy. Gundert attributes to Robert F. Kennedy, Jr. the claim that “research shows that exit polls are almost always spot on.” What research does RFK Jr. cite? Frankly, not much. He claims that “Exit polls in Germany… have never missed the mark by more than three-tenths of one percent.” And he quotes “political consultant” Dick Morris — some of us might use other descriptors — as saying that “Exit polls are almost never wrong.” That’s it, really.
Even with so few specifics, it’s kind of depressing to wade through the fact-check. Briefly: Even Kennedy’s own source didn’t claim that German exit polls “have never missed” by more than 0.3% (that was an average); in September 2005, they were off by 3 points. In the United States, things have often gone worse. In 1992, the average “within-precinct” exit poll error was 5.0 points in favor of Bill Clinton (i.e., overstating his vote share). (As Mark Blumenthal pointed out in January 2005, the classic documentary The War Room documents some of the erroneous projections in 1992.) Substantial discrepancies date back to at least 1988. Simply put, the myth of exit poll accuracy sprang into being after the 2004 election, although it was already knowably wrong.
2. The myth of intrinsic credibility. A common rejoinder to these points is: “How do you know the exit polls weren’t right in 1992 and those other elections you’ve mentioned?” (Duck the flying goalposts! It’s spooky how fast people can shift from “we know that exit polls are accurate” to “you haven’t proven that these exit polls are wrong.”) It seems to me that a better question is whether exit poll results have established their intrinsic credibility in cases where they diverge from official returns.
In a sentence: I don’t trust the 2004 Kerry/Bush exit poll results (see pp. 21-22) because, inter alia, they show Kerry winning Minnesota, New Hampshire, and Pennsylvania all by double digits. Those were hotly contested battleground states. (There are plenty of other bizarre results.) I don’t think any informed political observer believes those exit poll results. Exit poll fundamentalists generally don’t talk about them.
3. The myth of verification abroad. Gundert further quotes Dick Morris via Kennedy: exit polls supposedly are “so reliable” that they are “used as guides to the relative honesty of elections in Third World countries.” Yeah, not so much. Characteristically, Kennedy cites the examples of the Republic of Georgia in 2003 and Ukraine in 2004. Like every other example I know of, these elections provided plentiful and varied evidence of pervasive fraud, documented in extenso by international observers (Georgia, Ukraine). Exit polls were indeed conducted in these elections, but they weren’t central to the critique of either election. For what it’s worth, the Carter Center’s position is summed up in this 1994 recommendation for Mexico:
Avoid exit polls, which are unreliable in a climate of suspicion and which will create a negative atmosphere if the voters feel they are being watched.
The U.S. government nonetheless has funded exit polls in other countries, but has been careful not to rely on the results. “Exit polls reliably detect fraud because they work so well in Germany,” said no one ever — until RFK Jr. and his comrades came along.
4. The mystification of the irrelevant null hypothesis. Statistical analysis often proceeds by designating some “null hypothesis” — generally “no effect” or “no difference.” In exit poll discussions, generally the ‘null,’ stated informally, is that the exit poll is a random sample of votes from the official results: even less formally, that the exit poll results match the official results within random sampling error. Here Gundert says something basically true and important: “When the exit polls are way off” — if the null hypothesis appears to be wrong — “either the polls are wrong, electoral fraud was committed, or both.” Of course, the official results could be wrong for other reasons. But Gundert makes a crucial point: knowing that the exit poll results are “way off” from the official results doesn’t tell us why.
Exit poll fundamentalists, predictably, tend to butcher that point. Take Richard Charnin, of whom Tim Robbins writes, “I defer to his expertise.” In a May 5 article on his blog, Charnin states,
This is an analysis of estimated probabilities of fraud in the Democratic primaries…. The probability of fraud is calculated using the Normal distribution…. The total primaries average probability of fraud is 97.4%.
This is gibberish. In classic statistics, it’s simply impossible to calculate a “probability of fraud.” Charnin is misrepresenting the evidence that, in Gundert’s words, “the exit polls are way off” as evidence that the official results are wrong. He doesn’t even consider the possibility that the exit polls are wrong, apart from random error. (In Bayesian statistics one might calculate something like a probability of fraud. But to get “97.4%,” one would have to explicitly assume that the exit polls are accurate apart from sampling error. I’ve already explained why this assumption is reckless.)
In an April 23 post, Charnin goes even further, and weirder. Discussing the New York exit poll results, he says that adjustments made between 9 pm and the final results are “mathematically impossible” and: “Therefore, the recorded vote was also mathematically impossible. The impossible adjustments are irrefutable proof of election fraud.” If you’re wondering how exit poll adjustments ever could provide “irrefutable proof of election fraud” — yeah, they can’t. This isn’t something you missed in math class, or a class you never took. It’s just bullshit. When Robbins defers to Charnin’s “expertise,” he is Doing It Wrong.
So what?
If experience is any guide, some people will construe this article as a sweeping dismissal of any and all concerns about election fraud, SNAFU, vulnerabilities, and other such issues, or at least a sly effort to marginalize these issues. On the contrary, I strongly favor informed discussion about election issues. Tim Robbins titles his piece, “We Need to Fix Our Broken Election System” — and I don’t know anyone who fundamentally disagrees with that. But, ultimately, to accept exit poll bunk requires willful suspension of critical thinking, and I don’t expect that to end well.
By the same token, if you are predisposed to dismiss all complaints about elections as CT or “sore loser-ness,” watch out. We all have a shared stake in figuring out what is broken, and fixing it. That isn’t as easy as rehashing the primary wars, but it’s probably more useful.