Cooking the polls

So yesterday, in the midst of some friendly jousting between my friend Jerome Armstrong and I, he made a serious accusation against pollster Research 2000's daily tracking poll.

I will give R2K the "Zogby Award" for 2008. This is the award that goes to the pollster that, given the final result, is consistently off until the final days before the election, when they poll to make it more in line with the other poll projections. So congratulations, Markos, on your R2K success.

Let's look at the results, taking the results from 10/25, when the results were at its supposed "outlier" place, and the final one on 11/4 when the numbers had closed:

           10/25                    11/4

     Obama McCain Other/Und   Obama McCain Other/Und

DEM    88     7     5           88     10     2  
REP     7    87     6            5     93     2
IND    50    40    10           49     46     5

Now remember that R2K never messed with their sample composition -- something other pollsters did to try and bring their results more inline with other pollsters (like Battleground which started with an almost even partisan breakdown, before pushing it out to more realistic numbers). So if R2K was going to cook its numbers, it couldn't do so by simply boosting the number of white respondents or kicking in more Republicans or whatnot.

To cook the books, it would have to tweak the poll results themselves. But look at the numbers above.

What we saw happen is that 1) Republicans came home. No one doubted this. While some Republicans flirted with the likes of Obama and Barr, at the end of the day, they came home to their nominee, and 2) undecided independents came in for McCain.

Actually, in the 10/25 sample, 4 percent of the "independent" sample chose Barr, suggesting strongly that many of those "Independents" were really disaffected Republicans running away from their party's shitty brand. By November 4, that Barr support had evaporated. That movement toward McCain was really Republican-leaning "independents" coming home to McCain as well.

So for Jerome to claim that Research 2000 cooked its numbers, he would have to suggest that those numbers and trends were really out of the ordinary -- that Republicans weren't squishy on McCain, and that they didn't come home to him the final days as everyone expected. Of course, that would be patently absurd.

Jerome was a big critic of this poll throughout its run, partly because it was so transparent that he (and others) could nitpick the internals to death (no other poll was this transparent, hence they were spared the extra scrutiny). And I know that it must kill my friend that despite his constant dissing of this poll, that it was among the best this cycle, and ultimately performed better than his so-called gold standards -- the Battleground Poll and Gallup.

Yet, while there may be plenty to nitpick about the R2K poll (its Latino sample was way too optimistic), fact is it did pretty well, and to claim without any evidence that it cooked its books -- a la Zogby -- is pretty ridiculous.

All that said, I'm willing to let Nate Silver adjudicate this friendly dispute.

What do you think, Nate? Did R2K cook its numbers?