Skip to main content

I've seen a lot of hand wringing and out right hostility that follows a single poor poll result.  The clearest example as of late was the poll from PPP in Iowa that showed Romney +1.  People were quick to dismiss PPP as unreliable, call on Markos to quit using them as a firm, and even claim that they are in the tank with the conservative media trying to drive a narrative.  These same people pointed to the NBC poll in Iowa showing Obama +8 as being accurate and evidence that PPP was wrong, in the tank, etc. and so on.  All this comes from a naivete about how polls actually work and what numbers we should expect to see.  The most likely reality is that we are in fact winning Iowa in the low single digits and both polls are evidence that point to this.

What people fail to understand is that every single poll contains a margin of error.  The pollster himself is telling you that he isn't exactly sure of the number, but believes it is highly likely to be close to the exact number within a certain percent - the margin of error.

The actual PPP result for Iowa was Romney 49 +- 3.3 and Obama 48 +- 3.3.  What PPP was telling us, in other words is that Iowa was somewhere between Romney 46 to 52 and Obama 45 to 51, or anywhere from Obama +5 to Romney +6.  With NBC we had Obama 51 +-2.9 and Romney 43 +-2.9 giving a range of Obama +14 to Obama +2.

Notice that both these polls overlap each other in what they are actually telling us.  In fact if you take them together you have good evidence that NBC was a bit high while PPP was a bit low.  If we take the Pollster Iowa average of Obama +2.9 we see that it is within the probable range of both polls.  In fact the evidence between the two is that probably Obama leads by a bit more than 2.9% so these two polls together are driving his number up a bit in the Pollster average.

In a universe where Obama has a +3 advantage in Iowa, we would NOT expect to see poll after poll showing Obama +3.  If a poll happened to show +3 exactly, this has as much to do with luck as anything else.  In fact, because the margin of error is calculated to something like 95% in most polls you would expect to see about one out of every 20 miss completely and show a number whose error range does not actually include the real result.

I blame this misunderstanding on the media.  Think how less concern there would be if the PPP number had been reported as Romney +6 to Obama +5 rather than Romney +1.  The latter sounds misleading in its implied precision, especially since they don't usually mention the error bars.  When they do report the error range, the news media never emphasizes it and certainly doesn't explain it.

What all of this is saying is please look at the averages rather than the individual polls. It is all well and good to look at polls as they come out.  I certainly do it myself.  But when you do so just remember the error range and look at how they fit into the average.

Also don't get mad at a polling firm just because they report a result that happens to be at the low range of what is really going on.

12:01 PM PT: Please read this before you freak out about the +1 in Ohio that PPP posted.  A +1 in Ohio is very much in line with the likelihood that Obama has about a two to three point lead.


Do I feel better about single bad polls and polling firms such as PPP that release them from time to time?

55%22 votes
0%0 votes
17%7 votes
27%11 votes

| 40 votes | Vote | Results

Your Email has been sent.
You must add at least one tag to this diary before publishing it.

Add keywords that describe this diary. Separate multiple keywords with commas.
Tagging tips - Search For Tags - Browse For Tags


More Tagging tips:

A tag is a way to search for this diary. If someone is searching for "Barack Obama," is this a diary they'd be trying to find?

Use a person's full name, without any title. Senator Obama may become President Obama, and Michelle Obama might run for office.

If your diary covers an election or elected official, use election tags, which are generally the state abbreviation followed by the office. CA-01 is the first district House seat. CA-Sen covers both senate races. NY-GOV covers the New York governor's race.

Tags do not compound: that is, "education reform" is a completely different tag from "education". A tag like "reform" alone is probably not meaningful.

Consider if one or more of these tags fits your diary: Civil Rights, Community, Congress, Culture, Economy, Education, Elections, Energy, Environment, Health Care, International, Labor, Law, Media, Meta, National Security, Science, Transportation, or White House. If your diary is specific to a state, consider adding the state (California, Texas, etc). Keep in mind, though, that there are many wonderful and important diaries that don't fit in any of these tags. Don't worry if yours doesn't.

You can add a private note to this diary when hotlisting it:
Are you sure you want to remove this diary from your hotlist?
Are you sure you want to remove your recommendation? You can only recommend a diary once, so you will not be able to re-recommend it afterwards.
Rescue this diary, and add a note:
Are you sure you want to remove this diary from Rescue?
Choose where to republish this diary. The diary will be added to the queue for that group. Publish it from the queue to make it appear.

You must be a member of a group to use this feature.

Add a quick update to your diary without changing the diary itself:
Are you sure you want to remove this diary?
(The diary will be removed from the site and returned to your drafts for further editing.)
(The diary will be removed.)
Are you sure you want to save these changes to the published diary?

Comment Preferences

Subscribe or Donate to support Daily Kos.

Click here for the mobile view of the site