Skip to main content

View Diary: Toward meta-meta-analysis of election polls (30 comments)

Comment Preferences

  •  There is a basic flaw in combining (0+ / 0-)

    data using different methods - not only different mathematical procedures, but more importantly, how poll data are selected for inclusion.

    One way, as you're doing, is to hope that the law of large numbers or law of averages will cause these problems to become inconsequential.  But they are not.

    Polling firms introduce systematic bias into the data, and this can result in shifting the entire distribution % pts in either direction (usually to the political right, but depending on whether LV or RV are used).

    Nate tries to control for house effects, and he also has complex regression models not used by others.  Not sure he's most accurate, but he's doing something important, more like science & logic combined than pure statistics.  But his house effect controls are less than perfect, as admitted.

    BUT WE CAN AVERAGE THE NUMBERS IN OUR HEADS IF WE HAVE THE MEAN FOR EACH METHOD.  And the numbers aren't far apart.  NONETHELESS, THE AVERAGE OR EVEN THE RANGE COULD BE INACCURATE IN PREDICTING FINAL RESULT.

    Why?

    Because to model an election, one is modeling a dynamic process, not a static one.

    So the models of behavior today are not going to predict what happens in a week or two - - as we have seen, even a Senate candidate (Akin, Mourdock) can affect polling, or in 2004 the "Bin Ladin Tape" appears to have had a huge effect (Kerry was ahead in the polls on Friday, but polls shifted immediately after the Bin Ladin Tape was aired nationally.

    So, most importantly, it is the dynamics that are missing from the meta-meta-analysis.

    One approach to this problem would require comparisons of data models and outcomes over time, across multiple elections, and even then the findings would not generalize perfectly, as no two elections are alike.

    •  Actually... (0+ / 0-)

      ...the fact that the different models use different methods and make different assumptions, and yet none of them is probably quite right, is precisely the reason why I am interesting in averaging them. What do they have to say in aggregate? That is a useful question, and the answer is useful.

      Brash Equilibrium /brASH ēkwəˈLIBrēəm/ Noun: a state in which the opposing forces of snark and information are balanced

      by Brash Equilibrium on Fri Oct 26, 2012 at 04:15:19 AM PDT

      [ Parent ]

Subscribe or Donate to support Daily Kos.

Click here for the mobile view of the site