I don’t like cherry picking polls. I think that is prescription for re-enforcing your own prejudices. But this morning's USA Today/Gallup Poll demands comment. It is, to borrow a phrase from the New York Times, yet another lesson in the "perils of polling".
The first time serious questions were raised about Gallup’s tracking polls was 1996. Consider the following quote from a New York Times account in October of 1996 highlighting the absurd volatility Gallup was finding:
''The latest CNN/USA Today Gallup tracking poll shows the race nationwide is tightening,'' Wolf Blitzer, a CNN correspondent, said on the program ''Inside Politics'' on Saturday. ''This single-digit lead is half of what it was earlier this month.''
Appearing on the program, Scott Reed, the Dole campaign manager, said: ''That's consistent with our internal numbers. And what you've seen here in the last 10 days, the race has closed.''
But by Tuesday, Judy Woodruff, also on ''Inside Politics,'' declared that Mr. Clinton's ''domestic political standing is strong,'' announcing that he had opened his largest lead, 25 points, since the poll started at the beginning of September.
The true height of absurdity was Gallup’s tracking polls in 2000. They were laughable. Alan Abramowitz of Emory recently highlighted one such example: "On October 24 Gallup had Gore ahead of Bush by one point. Three days later, on October 27, they had Bush ahead by 13".
He went on to note:
No other poll during the 2000 campaign showed anything like the volatility of the Gallup tracking poll and so far Gallup's 2008 tracking poll has shown nothing like the volatility of their 2000 tracking poll. That's probably because they haven't started to apply their likely voter screen to the tracking poll.
A study by Robert Erickson of Columbia concluded:
This article explores how Gallup’s likely voter model exaggerates the reported volatility of voter preferences during the campaign. Much of the reported variation in candidate preference reported by Gallup in that election is not due to actual voter shifts in preference but rather to changes in the composition of Gallup’s likely voter pool. The findings highlight dangers of relying on samples of likely voters when polling well before Election Day.
Over the last year I have studied polling history a fair amount. One conclusion I have reached is that medians are far more useful than averages. In any cycle there are outliers, and the inclusion of those polls in averages will inevitably skew the results. I can point to other examples of this (Pew showed Bush with 16 point lead after the 2004 GOP convention when no one else found a lead larger than 11).
There can be no question that McCain has received a substantial bounce – and it's cause for concern. But the biggest concern I have right now is that two pollsters with questionable records are driving the narrative. Caution is appropriate in over reading polls right now.