Polls can raise hopes, polls can dash hopes. Polls lead to political decisions: Candidates drop out of or enter races based on polls. Hell, whether or not the candidate gets to debate the issues with his competitors depends on polls, at least for Republican presidential candidates in 2015. But polls suck, and few know that fact better than the the people who put together the Daily Kos Elections Morning Digest. The Digest has a lot to say about polling and is presented daily with the task of trying to parse the various polls and their inevitable disagreements. This morning, the Digest said in part: “At this point, though, we're going to say something you probably never expected we'd say: The only poll that matters is on Election Day. Have we lost our marbles? Not hardly—it's the political world that's gone crazy. For starters, we've been burned a lot by wrong-headed red-state polling lately, and we aren't just talking about Kentucky earlier this month...We also saw how badly polls performed in many other states last year: Kansas, Arkansas, Kentucky (again), Georgia ... the list is distressingly long.”
In 2014, the polls were off everywhere. The great Democratic comeback turned into a rout on election day. Below are some possible reasons why polls are bad for all of us. Remember, anyone can conduct a poll on any subject and publish the results, and that may be part of the problem.
Although some might disagree, The New Yorker magazine is not just about entertainment, cartoons, and weird fiction. In the November 16 issue there is a piece by Jill Lepore titled “Politics and the New Machine” much of which looks into the history of polling all the way up to its present iterations; it is well worth reading in full. (And while you're there, check out the letter to Daily Kos in Shouts and Murmurs on page 45!)
Lepore reminds us of the monumental goofs of polls over the years, beginning with George Gallup's prediction of Dewey's win over Truman in 1948, and Eisenhower's defeat of Stevenson in 1952. She also discusses in depth some of the philosophical issues involved in the whole concept of polling.
Lepore also points out, with a quote from a former managing editor at Gallup, that the increasing number of polls by media, including newspapers and television news, lead to both “distorted readings of the electoral climate” and “manufacture a false public consensus on policy issues.”
As a social scientist, I would like to clarify and expand on this conversation. For example, a person conducting an opinion survey ( which is what a poll is) can virtually guarantee any outcome he or she desires, simply by the way the questions are worded. For example (until they finally wised up), I used to get frequent surveys from the Republican Majority Leader and the RNC in which they would ask questions like: Do you prefer the failed tax and spend policies of the Democrat (no letter s) or the responsible leadership of the Republican party? Imagine a three page questionnaire with choices like that.
A second point to make is this: Whenever you see a poll you should also see, “The margin of error is plus or minus...” This figure is usually around 3 or 4 with a decimal point somewhere. Please know that this is a statistical number only, which is based on the sample size. For example, the larger the sample size, the smaller the margin of error. It has nothing to do with the quality or purpose of the survey.
A third point is based on what is called response rates. Since practically all polls are done by phone or (recently) by computer---which has an even more problematic selection issue---what do you do if you have a list of 10,000 phone numbers of people in your target demographic and the majority of the people you call don't answer, or hang up when you tell them why you're calling? Who are the people who hang up, and why did they hang up? How about those who aren't home? We don't have those answers and it may well be that had we been able to survey these people, their answers may well have changed the results of the poll. It's public knowledge that response rates on surveys have dropped below ten percent of respondents. Does this low response rate have an effect on survey results? Probably, but we don't know for sure.
And finally, here is the biggest gotcha of all. What about the low-information voters, the ones who are unaware of the issues or the candidates? You ask, who do you prefer, candidate Stinky or candidate Smelly? The respondent thinks, well, being stinky is probably worse than being smelly, so I'll say Smelly. Now let's reword the question.”Who do you prefer, Republican Stinky or Democrat Smelly? Now the person has some information to work with; he knows he's a Republican or democrat, so the answer gets much easier.