Skip to main content

View Diary: The best pollsters of 2006 (77 comments)

Comment Preferences

  •  OK on accuracy, but how about bias? (2+ / 0-)
    Recommended by:
    billlaurelMD, epppie

    Was their a directional bias to the errors?  I say this because I had the impression that Rasmussen's errors were predominantly in a direction that favored Republicans.  It's great that they got within 4 points on average, but if the 4 points tended to be predominantly off in one direction, that's important because it means that the poll in question has a differential bias, however small.  If they have such a bias that they can't control, don't understand well enough to adjust for, that means that next cycle we won't know until after the election whether demographic change has made the bias so large that the poll is no longer reliable.

    •  Mason-Dixon was the most biased firm (0+ / 0-)

      In every Senate race where they didn't peg the numbers exactly, M-D polls favored the Republican candidate--sometimes by huge margins (overestimated Chafee by 7 in RI, overestimated Corker by 9 in TN, overestimated Steele by 7 in MD, etc).

      There was a single exception: Arizona, where they overestimated Pederson by one point.

    •  You're correct (0+ / 0-)

      It's my impression that Rasmussen does a lot of work for corporate/Republican/conservative sponsors. (I'm by no means certain that is true, but I hope someone will correct me if I'm wrong.)

      Based on reading their material for several years, I'm inclined to think that in general Rasmussen results favor Republican/Conservative positions (both in election and public opinion issues) by a 3%-5% tilt.

      It's difficult to tell, however, whether the bias comes about as a result of sampling error, their polling methods, or the models Rasmussen uses to gauge "likely voters" at least in their election polling.

      In any event, while Rasmussen regularly crows about their accuracy, I suspect that the staff is looking carefully at the factors that led them to consistently underestimate the Democratic support in race after race.

      •  Less worrisome if intentional (0+ / 0-)

        If Rasmussen is simply doing its adjustment of the raw data on purpose so as to make it look 3-4 points optimistic for their Republican clientele, that's not so concerning.  In this explanation, they are in control.  They know where the bias comes from, and simply choose to let a little in to help their ability to keep their clients happy.  They presumably wouldn't let the gap from reality get too big, because while being the bearer of good news is nice for customer relations, being wrong too often, beyond your quoted margin of error, is really bad for business.  If they are introducing a bias intentionally, we can all just mentally subtract 3-4 points from what Rasmussen tells us the Republican candidate is doing, and carry on.

        The real problem is that Rasmussen, and all the rest, can't control all the biases introduced by the potentially very profound ways that the population of folks they can get to answer their surveys differs from the population of folks who are actually going to vote.  You can only adjust for factors you know are important and can identify, such as age, gender, ethnicity, and socioeconomic status (party identification is controversial), but you can't adjust for factors you don't know are important, and therefore didn't or couldn't determine when you did your survey.  Recently, the profound selection bias built into a system in which, reportedly, as much as 90% of the folks they intend to survey can't be reached or refuse to participate, when adjusted for all the known factors like age, gender, etc., has happened to, mostly, produce samples whose answers coincide pretty well with the actual results.  If Rasmussen and M-D are not intentionally cooking the data to look mildly over-Republican, but really are getting results that systematically skew 3-4 points Republican because the coincidence of biases, being a chance coincidence, isn't perfect, that means that they have no control over the magnitude of this bias, because it arises from factors they don't understand.  Just because the magnitude is 3-4 points this season, the fact that the cause is unknown means we don't know that it won't be 4-5, or 9-10 points next cycle.  We wouldn't know until election night.

Subscribe or Donate to support Daily Kos.

Click here for the mobile view of the site