I've been meaning to write about this for a while, but in the craziness of the election it seemed like it would get stampeded by more pressing concerns. We owe an enormous debt to folks like Nate Silver at fivethirtyeight.com for helping us understand what's going on in when new polls come in and how to interpret them in the broader context of the totality of polls.
One aspect that's been missed, and I believe is critical, is understanding the direction of causality in respondents' answers to questions after the topline numbers.
Take a very familiar type of analysis: 65% percent of Obama voters say that the Economy is the most important issue in the election, while 58% of McCain supporters say their most important issue is Terrorism (note: these are hypothetical made up numbers). It is reasonable to infer from this that people who care about the Economy more tend to vote for Obama, and people who care about Terrorism more vote for McCain. But it's not necessarily true. It could just as easily be the case that people who vote for Barack Obama end up caring about the economy more -- partly because it's a central theme of his campaign, and an issue in which there's a very distinct difference between the candidates. People could be voting for Obama for all kinds of reasons, but when asked why, the Economy is the answer that comes to mind that best explains their vote.
Why is this important?
Take the flip side: the McCain voters who claim they care about Terrorism. Do they really care that much about terrorism -- or is terrorism one of the few things they can cite and still feel justified in voting for McCain? After all, they can't easily cite the Economy, when their candidate has claimed not to know much about it and seems to be lagging behind his Democratic opponent on the issue.
So the question becomes, for a Democratic candidate: if I were to take a strong stand on Terrorism, would a whole bunch of the voters who cited Terrorism as a concern switch sides. Maybe. But quite likely not. These are McCain voters who are simply pinning their support on Terrorism; if you took away terrorism as an issue they would simply move on to the next thing - abortion, gay marriage, or earmarks.
This is all a question of causality. Do people support McCain because they care about Terrorism, or do they [say they] care about Terrorism because they support McCain? It's probably a healthy dose of both in this case -- but it's usually interpreted in just one direction.
Take the story on the front page right now, about Sarah Palin. Apparently 91% of McCain voters think she's qualified to be president, while 82% of Obama voters think she isn't. Is that because these two groups of voters have different criteria by which they judge a person's qualifications to be president? Perhaps that's part of the answer. But do you really think that if Obama had nominated a Democratic version of Palin (hard to imagine, true, but just play along), don't you think all of a sudden Republican voters would adopt much more stringent criteria, and many Democrats would loosen their standards a little?
When you see cases of "IOKIYAR" (It's OK If You Are Republican), you are seeing the same issue at play. Republicans think lying under oath is the worst thing in the world when Bill Clinton does it; but when it's Scooter Libby, then it's OK. And Democrats are not immune to this bias either. People's opinions are constrained by their other commitments, whether to a candidate, a party, an ideology, or to any number of other things.
So when you read about respondent's opinions in a poll, ask yourself this question: is that really what they believe, or is that what they have to believe in order to justify their broader opinions and ideology?
----
Let me end with a quiz -- there's no right answer here, so don't worry :)
During the Democratic primary, Hillary Clinton proposed a gas tax holiday and Obama opposed it. There were issues on both sides (I'll leave my personal opinion out of it), but critically, a large majority of Clinton supporters liked the Gas Tax Holiday idea, and a similar proportion of Obama supporters opposed it.
So here's the question: Was it the case that these were people's "objective" opinions, and it just so happened that people who liked the Holiday also liked HRC's other positions, and vice versa for BHO supporters? Or was it the case that people's opinions were informed in part by whether their preferred candidate supported or opposed the idea?
In the poll below you get an either or choice to make a point, though I will grant that it's not an either or question -- it can be, an usually, some of both. So I guess the question is, which is more true?