Skip to main content

View Diary: Do states have 'house effects' when it comes to polling? (68 comments)

Comment Preferences

  •  Meta polling (12+ / 0-)

    In the end, isn't meta polling just increasing the sample size of the polls? That would seem to make the results more reliable. If 10 firms each poll 1000 people and we average the 10 polls would the results really be that different from originally polling 10,000 people?

    To me progress is not so much a goal as it is a process and I believe it will not follow a straight course. Remember, the drops of water that form the river may not take the shortest path but they will still reach the ocean.

    by ontheleftcoast on Sun Dec 23, 2012 at 02:14:43 PM PST

    •  Not that different (3+ / 0-)
      Recommended by:
      radarlady, llywrch, ontheleftcoast

      Actually usually worse than getting a true 10k sample, since the ten firms will likely poll some of the same people.

      But a truly random distribution of 10k is MUCH MUCH more significant than 1K in terms of margins of error.

      The worry with modern polling is the knowledge that the sample is NOT random, and the various models used to estimate the fraction that doesn't answer polls for whatever reason seem to have issues (this article is saying that in 2012, all polls seemed to come in low for the actual winner.  That's a fairly strange bias and thus interesting to study)

      •  Non random samples - why ten 1K polls are better (1+ / 0-)
        Recommended by:

        If you had ONE 10K sample - whatever bias was in the selection would go unbalanced.  By having TEN 1K polls, sample bias is balanced out (as long as you don't have lots of intentional bias, a la Razzy and Gravis)

        •  Only if the bias isn't in the same direction (0+ / 0-)

          You can't assume that.

          Most polling firms try pretty hard on the sampling (the likely voter screen is more likely to have partisan bias).  The systematic errors are far more likely to be tied to methodology than to an attempt to skew the outcome.

          Because of that, because the sampling methodology is similar between polling firms (the primary differences are robocalling vs live interviewer and whether the interviewer can handle languages other than english) there is a concern that even a larger sample won't be free of bias (multiple poll averaging won't eliminate errors, they'll just reinforce them)

      •  Oh lord no. This is dead wrong. (0+ / 0-)

        Sample size has strongly diminishing returns. Standard deviation of a poll, where p is how many people chose candidate A and n the sample size, is sqrt(p*(1-p)/n). That should make fairly obvious that one large poll performs worse than several small ones-- the error doesn't shrink proportionally, it's proportional to the square root.

    •  This depends (3+ / 0-)

      If it were possible to get a truly random sample of voters, and if pollsters managed this (or came close to it) then it would not matter much, if at all.

      However, the above is not possible. Not only is each pollster's sample biased, but each pollster uses a different weighting scheme to try to eliminate this bias.

      In these circumstances, combining 10 different pollsters may be a good bit better than having one pollster get 10 times the sample.

      Increasing the sample reduces the variance.

      Increasing the number of pollsters may reduce bias

      (Here I am using bias in its technical sense - results that are systematically different from the true value - and do not mean to imply any nefarious intent).

    •  Not entirely an increase in sample size (1+ / 0-)
      Recommended by:
      Steve Singiser

      If that were the case, then we would have predicted a near-tie in the popular vote based on on national polls. But I did not predict that - I predicted an Obama popular vote win by several points.

      Sampling error is just part of the story. One needs to have to have a way to address systematic errors in polling.

Subscribe or Donate to support Daily Kos.

Click here for the mobile view of the site