Skip to main content

One of the terrific features that first introduced to the polling junkies a few years back is the ability to remove pollsters from their aggregate (TPM's Poll Tracker gives you the same wonderful feature, but Pollster did it first).

It's a feature I have used ever since it was available, and this year it's especially fun to do. For example, this is just Rasmussen and Gallup (aka "the trackers," short for daily tracking polls):

And this is everyone else:

Notice any difference? I did, and it bothered me.

When polling data doesn't make sense, I turn to the experts. In this case, here are some important observations by Mark Blumenthal and Alan Abramowitz on the tracker conundrums:

Polling Party Identification: Rasmussen and Gallup vs. Everyone Else (Alan Abramowitz, Daily Kos)

Why has Obama Taken the Lead in the Gallup Poll? (Alan Abramowitz, HuffPost's

Gallup Poll's Race Issues Should Be Singled Out (Mark Blumenthal, HuffPost's

Race Matters: Why Gallup Poll Finds Less Support For President Obama (Mark Blumenthal, HuffPost's

I understand that not everyone will agree, but the other feature of the polling graphs is that they are clearly marked as "customized," and you can easily add the trackers back in if you so desire. The action is transparent.

When there are better explanations for the discrepancies from the pollsters, I will add them back. I will appreciate feedback about new data that should make me reconsider. Until then, the aggregates sans Rasmussen and Gallup will be my standard.

Your Email has been sent.
You must add at least one tag to this diary before publishing it.

Add keywords that describe this diary. Separate multiple keywords with commas.
Tagging tips - Search For Tags - Browse For Tags


More Tagging tips:

A tag is a way to search for this diary. If someone is searching for "Barack Obama," is this a diary they'd be trying to find?

Use a person's full name, without any title. Senator Obama may become President Obama, and Michelle Obama might run for office.

If your diary covers an election or elected official, use election tags, which are generally the state abbreviation followed by the office. CA-01 is the first district House seat. CA-Sen covers both senate races. NY-GOV covers the New York governor's race.

Tags do not compound: that is, "education reform" is a completely different tag from "education". A tag like "reform" alone is probably not meaningful.

Consider if one or more of these tags fits your diary: Civil Rights, Community, Congress, Culture, Economy, Education, Elections, Energy, Environment, Health Care, International, Labor, Law, Media, Meta, National Security, Science, Transportation, or White House. If your diary is specific to a state, consider adding the state (California, Texas, etc). Keep in mind, though, that there are many wonderful and important diaries that don't fit in any of these tags. Don't worry if yours doesn't.

You can add a private note to this diary when hotlisting it:
Are you sure you want to remove this diary from your hotlist?
Are you sure you want to remove your recommendation? You can only recommend a diary once, so you will not be able to re-recommend it afterwards.
Rescue this diary, and add a note:
Are you sure you want to remove this diary from Rescue?
Choose where to republish this diary. The diary will be added to the queue for that group. Publish it from the queue to make it appear.

You must be a member of a group to use this feature.

Add a quick update to your diary without changing the diary itself:
Are you sure you want to remove this diary?
(The diary will be removed from the site and returned to your drafts for further editing.)
(The diary will be removed.)
Are you sure you want to save these changes to the published diary?

Comment Preferences

  •  Tip Jar (157+ / 0-)
    Recommended by:
    Remediator, Texknight, kestrel9000, Blue in VA, AZDem, Crashing Vor, redstella, tampaedski, KroneckerD, philipmerrill, AAMOM, Mary Mike, elziax, skohayes, Jeremy10036, oxfdblue, mama jo, TheGreatLeapForward, JanF, TFinSF, cv lurking gf, MKSinSA, Floande, Dobber, Temmoku, Melquiades, We Won, gulfgal98, leu2500, pamelabrown, gchaucer2, camlbacker, BachFan, lizah, ExStr8, kerflooey, EcosseNJ, skillet, Emerson, thankgodforairamerica, Pinto Pony, hyperstation, followyourbliss, MRA NY, terjeanderson, indie17, Alma, tomephil, middleagedhousewife, Gemina13, litho, Ice Blue, Andrew F Cockburn, MartyM, Timaeus, dougymi, MKS, radarlady, Joy of Fishes, Nowhere Man, buckstop, FWIW, belinda ridgewood, nomandates, OldGrammy, joedemocrat, FlyingToaster, 57andFemale, sngmama, Its a New Day, zerelda, Supavash, IndieGuy, zukesgirl64, KiB, AreDeutz, US Blues, Bush Bites, tin woodswoman, aufklaerer, Hey338Too, tgrshark13, Only Needs a Beat, RokkiBlueboa, Texdude50, cocinero, maybeeso in michigan, Sharon Wraight, Little Flower, askew, captainlaser, exterris, implicate order, revsue, IreGyre, bythesea, cotterperson, Gary Norton, Catte Nappe, yawnimawke, StellaRay, glorificus, Jim R, Scout Finch, Cronesense, DoctorWho, smartdemmg, cybersaur, johnosahon, Gustogirl, Meteor Blades, ratcityreprobate, quill, kalmoth, ParkRanger, geordie, doroma, steamed rice, Quite Contrary, wayoutinthestix, Iron Spider, cpresley, frostieb, Trotskyrepublican, Gator Keyfitz, Puddytat, bnasley, itskevin, Anthony Page aka SecondComing, pyegar, Delilah, elwior, annieli, edsbrooklyn, DeminNewJ, 2thanks, jgilhousen, Lahdee, BlueMississippi, pgm 01, doingbusinessas, Larsstephens, greycat, sparkysgal, pat208, Dutch Doctor, haremoor, MKinTN, harlinchi, deerang, a gilas girl, SadieSue, science nerd, browneverywhere, WiddieDawg, abarefootboy, PurpleThistles

    "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

    by Greg Dworkin on Sat Aug 18, 2012 at 05:07:56 AM PDT

  •  Thanks (20+ / 0-)

    I'd like to see 2 charts for the 'outliars'  here - 1 for Gallup, the other for Ras

    Thanks - I love the

  •  I've been doing it for a while too (20+ / 0-)

    The wild Ras swings are just odd, like yesterdays Wisconsin Senate poll. An 18% swing?? Even if it's in our favor I think it's bullshit...that's how unreliable they are.
    It's odd that no one there ever thinks for a minute "hey, doesn't this look a little screwy?" before hitting the 'send' button. I mean, if I had to put my name on a piece of work that doesn't make sense to anyone else in my field I'd be really embarrassed.

    Gallup is less volatile but it seems to have an imbedded demo skew that they can't or are unwilling to adjust.

    Divide and rule, a sound motto. Unite and lead, a better one. ::Goethe::

    by Jeremy10036 on Sat Aug 18, 2012 at 05:36:09 AM PDT

    •  Talk about screwy... (2+ / 0-)
      Recommended by:
      llywrch, elwior

      Check this out - Wisconsin Pres, all pollsters, with Adults, Reg Voters, and Likely Voters:

      Obama 48.8 and Romney 46.3

      Ditch Rassmusen and count only Likely Voters:

      Obama 50.7 and Romney 45

      Romney, now with Ryan on his coattails, should be winning Wisconsin handily (and he is, according to aggregate polling excluding Rassmussen).  But what the accurate polls are showing is Obama over the 50% benchmark.  

      It appears that what Rassmussen does is keep Obama below 50% on any poll (including favorability), and in places Romney slightly ahead, so that conservatives can point to "the recent polling suggests..." talking point.  Voters like to be seen as voting for a winner, but if their guy is always behind, it's discouraging for them.

      It's pretty disgusting to see every day a poll by Rassmussen that the pundits refer to...and it's off by three points at least in favor of Romney-Ryan.

      If I write something even barely above the threshold of human intelligence, it will be interpreted by Republicans as genius.

      by MoronMike on Sat Aug 18, 2012 at 09:52:55 AM PDT

      [ Parent ]

  •  Ras With Their Swings, Couldn't Guess; Gallup (9+ / 0-)

    maybe is tracking likely-to-show-up-and-be-permitted voters?

    We are called to speak for the weak, for the voiceless, for victims of our nation and for those it calls enemy.... --ML King "Beyond Vietnam"

    by Gooserock on Sat Aug 18, 2012 at 05:37:54 AM PDT

  •  In my government classes... (23+ / 0-)

    One of the most important lessons I feel I teach is about polling.

    I beat it into the kiddies (high school seniors) head that there is a lot more to the "headline number," and how important it is to look at the percentages of the different kinds of people they spoke to, etc.

  •  I did not realize that about the Gallup Poll. (20+ / 0-)

    Thanks for the explanation, DemFromCT. We certainly don't want polls to give us a rosy view of the race but we also don't want inaccurate polls to dampen enthusiasm. I know it depresses the hell out of me to think that our country could actually elect Mitt Romney.

    A nice 3 to 5 point lead for President Obama keeps just the right amount of heat on us GOTVers.

    Words have meaning. Our words will reflect what is in our souls.

    by JanF on Sat Aug 18, 2012 at 05:43:21 AM PDT

    •  I encourage people to look at all the polls (9+ / 0-)

      and I keep in mind Charles Franklin's admonition (co-founder, not to cherry pick and to include all the polls. You can aways add them back.

      However, it also bothers me that ras and gallup poll so frequently/daily compared to everyone else that their numbers have outsized influence.

      "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

      by Greg Dworkin on Sat Aug 18, 2012 at 06:11:09 AM PDT

      [ Parent ]

    •  When you look at polls (0+ / 0-)

      it is worth remembering that if they were accurate and reliable, we wouldn't need an election :)

      I am not saying that they have no value, because that would be a silly thing to say .... but be skeptical.

      I hope that the quality of debate will improve,
      but I fear we will remain Democrats.

      by twigg on Sat Aug 18, 2012 at 12:42:29 PM PDT

      [ Parent ]

  •  Then you should remove PPP also (7+ / 0-)

    I don't see a problem removing polls that are biased against Dems (for whatever reason), but the PPP polls lean the opposite way and should be removed also.  PPP also has the problem of polling registered voters, rather than likely voters, in some cases.

    Nate Silver tries to adjust for the bias, on all 3 of these polls, finding about 5% for Rasmussen.

    •  Nate also adjusts for the economy (12+ / 0-)

      and lots of other little things. I love his work, but don't foolishly try to duplicate it.

      If you want to add them back, or remove PPP, go ahead. In fact, you can remove all robo pollsters should you wish. Or you can remove just the internet pollsters. The charts are very flexible.

      it isn't D bias i am concerned about. I don't believe rasmussen even when he says Ds are doing well. I just don't buy his methodology.

      Gallup is honorable and experienced. I just don't fully understand why they stand out as being so different.

      "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

      by Greg Dworkin on Sat Aug 18, 2012 at 06:14:20 AM PDT

      [ Parent ]

      •  For me (1+ / 0-)
        Recommended by:

        I take out Ras, as they are dishonest and their numbers are simply not reliable.

        Removing Gallup, OTOH I think is a mistake.  Yes, they seem biased republican, but they are "honorable" as you say.  It just seems to me that if you take out Gallup, you also need to remove a corresponding liberal bias pollster.

        We remove Ras cause they are frauds, not because they skew republican. But removing Gallup seems a far harder sell to me.

        "Empty vessels make the loudest sound, they have the least wit and are the greatest blabbers" Plato

        by Empty Vessel on Sat Aug 18, 2012 at 07:55:07 AM PDT

        [ Parent ]

        •  More than seem (4+ / 0-)

          Gallup is biased Republican.

          NPR has one of the top Gallup honchos on every several days, and the guy has dropped Republican / Pundit Village talking points many times.

          "What could BPossibly go wrong??" -RLMiller "God is just pretend." - eru

          by nosleep4u on Sat Aug 18, 2012 at 08:26:39 AM PDT

          [ Parent ]

          •  Gallup was bought by a Repulican operative (5+ / 0-)

            several years ago.   They simply wanted the name -- it would be like BBC being quietly taken over by Fox News.  

            •  they are buying up everything... CNN went right (3+ / 0-)
              Recommended by:
              cybersaur, madhaus, Trotskyrepublican

              starting with the Time Warner merger in 1996... slowly at first but speeding up more recently... WaPo has a right wing owner now, but the change is still in its early days. The NY Times is still "liberal" but with Billionaires like Carlos Slim buying into it the shift seems inevitable... And don't forget already conservative papers going editorially even farther right like the Wall Street Journal...

              So for the plutonomy buying up their own prestige polling outfit goes right along with all of the rest. The radio networks all hard right or holy roller.... (same thing really) and all very handy to have along with the conservative owned companies who make and service the voting machines.... this is a vertical integration plutocracy... controlling opinion, news, votes, ID, education... wall to wall reality shaping to suit our new nobility.

              Pogo & Murphy's Law, every time. Also "Trust but verify" - St. Ronnie (hah...)

              by IreGyre on Sat Aug 18, 2012 at 09:16:37 AM PDT

              [ Parent ]

          •  Gallup 2010 performance atrocious (1+ / 0-)
            Recommended by:

              Remember the poll Gallup dropped right before the 2010 elections, where they said Republicans +15 nationally, with the possibility of a landslide of epic proportions.

               Final number was GOP by less than 7. Gallup was off by more than 8 points. That's nearly impossible to do by accident. They are Raz with a better name, and nothing more.

            •  whoa there (1+ / 0-)
              Recommended by:

              Generic House polls inherently aren't all that reliable, because -- weird as it is for Kossacks to think about -- many voters aren't really aware whether their incumbent is Democratic or Republican.

              Given that, R+15 in a generic House poll doesn't really translate into R+15 in the voting booth. Ergo, it isn't fair to Gallup to say that it was off by over 8 points on the margin.

              I'd need a lot better evidence than that to be convinced that Gallup deliberately cooked the poll.

        •  I think part of the problem is (2+ / 0-)
          Recommended by:
          DemFromCT, EcosseNJ

          right now the trackers are drowning the other national polls.

          At the moment, eliminating the other robocallers (PPP and Survey USA) as well as Gallup and Ras actually increases Obama's Pollster lead by about 1 point.

          Also, it's important to DemFromCT's approach that it uses the tools already available on the Pollster website, so that people can easily try alternative approaches. So some other ways that one might try to deal with the Gallup issue aren't readily available.

    •  btw, (2+ / 0-)
      Recommended by:
      pollwatcher, joedemocrat
      The Rasmussen Reports daily Presidential Tracking Poll for Friday shows President Obama attracting support from 46% of voters nationwide, while Mitt Romney earns the vote from 45%. Three percent (3%) prefer some other candidate, and six percent (6%) are undecided.
      Obama lead? I am still not using it.

      "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

      by Greg Dworkin on Sat Aug 18, 2012 at 06:20:57 AM PDT

      [ Parent ]

      •  Absolutely. (0+ / 0-)

        Once they've shown to be so far outside the rest of the polls, you can't use them even if they show a poll favoring Dems.

        And I fully agree with the methodology problems of Gallup.  It's a good organization, but their poll just seems weird sometimes.  The worst thing that could happen to Gallup is if they end up with a good final poll and they think they're methodology was accurate all along.

      •  The House of Ras (4+ / 0-)

        shows Obama up by 2 today.

        The Gallup uses 7 days so last Saturay and Sunday, the best of the Ryan roll-out is still being counted.

        Ras, has the traditional three days average, so the Ryan Honeymoon days have long since burnded off.

        Ras can show trends, and Obama will do no worse than Ras.  If Ras shows Obama up, Obama is most definitely leading.

        •  even so, if gallup comes back to even (1+ / 0-)
          Recommended by:

          as it has been, I still question it.

          "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

          by Greg Dworkin on Sat Aug 18, 2012 at 07:33:24 AM PDT

          [ Parent ]

          •  The polling seems static right now (2+ / 0-)
            Recommended by:
            Gator Keyfitz, Delilah

            The Purple Polls were taken on the two days including the Saturday of the Ryan roll-out.  So, the Virigina poll in particular showing Romney up by three there on the day of the big Ryan rally in Virgina would be somewhat suspect.

            I think the polling is okay.

            The money race does concern me.  

            I got two emails from Obama stating "This is critical" we are being outspent 2-1 in Iowa.   Did you get those?  That and Markos's diary about being outspent last week 3-1 spooked me.  I guess it worked because I donated three separate times.

        •  Ras doesn't even show trends. Their polls swing (0+ / 0-)

          harder than saturday night at the Apollo. They are just plain dishonest and validating them by accepting them only aids Scott Rasmussen and the GOP.

          If I knew it was going to be that kind of party, I'd have stuck my ---- in the mashed potatoes! - Paul's Boutique

          by DoctorWho on Sat Aug 18, 2012 at 09:36:46 AM PDT

          [ Parent ]

      •  The reason Obama is leading is clear (0+ / 0-)

        Convention bump for Romney will be greater.  I am pretty sure the same thing happened 4 and 8 years, but I am not positive.  

        I suspect one could find a correlation between Gallup and Ras polls throughout the year in comparison to polls in other cycles.   They just take a real poll about a week out to save face.  

        I think the standard should be transparent methodologies.  If they don't want to share, they shouldn't be taken seriously.

        But I am still interested in what they are telling the lemmings.

        "But once John Boehner is sworn in as Speaker, then he’s going to have responsibilities to govern. You can’t just stand on the sidelines and be a bomb thrower." - President Obama, 12-07-2010

        by justmy2 on Sat Aug 18, 2012 at 08:27:23 AM PDT

        [ Parent ]

        •  It did. some polls pre & post St Paul '08; (1+ / 0-)
          Recommended by:


          post DNC & during RNC in St Paul;
          8/29/2008    47.7    43.8
          9/01/2008    48.8    44.3
          9/06/2008    46.6    44.2

          post St Paul:
          9/08/2008    45.4    48.3
          9/17/2008    45.7    45.7
          9/18/2008    47.1    45.2

          "Never trust a man who, when left alone with a tea cosy, doesn't try it on!!"

          by EcosseNJ on Sat Aug 18, 2012 at 08:41:45 AM PDT

          [ Parent ]

        •  that's not clear at all (2+ / 0-)
          Recommended by:
          HudsonValleyMark, Delilah

          the convention bump usually includes the VP choice made just before or at the convention. No one knows what this year will bring, though there seem to be less undecideds than in previous years.

          One way to look at it is to take the D bounce and subtract the R bounce.  Since 1996 it's been 1-3 points. And 1996 was a weird year because Perot dropped out during the D convention, boosting Clinton. Don't look at that, look at the intervening years from then to now.

          "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

          by Greg Dworkin on Sat Aug 18, 2012 at 08:43:49 AM PDT

          [ Parent ]

    •  If PPP leans Dem we're in trouble (1+ / 0-)
      Recommended by:
    •  PPP is ALL likely voter now (1+ / 0-)
      Recommended by:

      They switched a couple weeks ago.

    •  PS. And I'm not sure why you'd imply (1+ / 0-)
      Recommended by:

      that PPP has been inconsistent in its methodology. They poll for registered voters until an election gets closer, when they switch to likely voters. That's the way polling is supposed to work. In fact, you could argue that PPP switched to likely voters too soon, as it's harder to get a good handle on who likely voters will be until the campaign has been fully engaged. Many organizations haven't switched over yet.

  •  Thank you for the APR........ (10+ / 0-)

       ...Please allow me this chance to thank you for the APR each morning. It's one of the reasons I check teh Kos every morning. Great work, and very much appreciated!

    Best, Hoghead99

    Compost for a greener piles?

    by Hoghead99 on Sat Aug 18, 2012 at 05:59:21 AM PDT

  •  I was wondering about that too. (1+ / 0-)
    Recommended by:

    Thanks for clearing it up visually before I could even ask! I figured this was the reason, But im glad to have it confirmed.

    Prinny bombs for my Prinny friend. Prinny dood meets a Prinny end.

    by kamrom on Sat Aug 18, 2012 at 06:16:51 AM PDT

  •  note the screen shot below (12+ / 0-)

    the variable charts in the post may change should you read this days or weeks later.

    This screen shot preserves what the charts looked like the day of the post.

    "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

    by Greg Dworkin on Sat Aug 18, 2012 at 06:28:11 AM PDT

    •  Since not everyone will read down this far (1+ / 0-)
      Recommended by:
      Calamity Jean

      I think it might be a good idea to include these in the diary -- these are, after all, the reason for writing the diary in the first place.

      Let us all have the strength to see the humanity in our enemies, and the courage to let them see the humanity in ourselves.

      by Nowhere Man on Sat Aug 18, 2012 at 07:55:41 AM PDT

      [ Parent ]

  •  Radio Wingnuts: According to Rasmussen, the most (0+ / 0-)

    reliable polling outfit out there, Romney is kicking ass.

  •  Why not show two graphs? (1+ / 0-)
    Recommended by:
    Joy of Fishes

    Everyone else and then Gallup and Rasmussen?

    Just a question. I would be interesting to track both as the election nears.  But I know that would be double the work.

    "A pride of lions" "A murder of crows" "A wunch of bankers"

    by Glinda on Sat Aug 18, 2012 at 06:45:05 AM PDT

  •  I got into it with a troll yesterday (0+ / 0-)

    who was insisting Romney didn't have to release his taxes but that Reid did need to reveal his source, and at one point the troll cited the Gallup poll because, um, well, just because.  I pointed out, citing DemFromCT, that Gallup isn't considered too reliable around here...

    Give me your tired, your poor, Your huddled masses yearning to breathe free
    ¡Boycott Arizona!

    by litho on Sat Aug 18, 2012 at 07:03:10 AM PDT

  •  Say "without" instead of "sans," fancy pants (1+ / 0-)
    Recommended by:

    I think there will probably 10-25% of readers who will stumble on "sans."

    See the losers in the best bars, meet the winners in the dives -Neil Young

    by danoland on Sat Aug 18, 2012 at 07:05:45 AM PDT

  •  By the way, the chart doesn't show up (0+ / 0-)

    on my Internet Explorer browser at work. I think there's some filter that stops it. Perhaps it's because it's active content.

    See the losers in the best bars, meet the winners in the dives -Neil Young

    by danoland on Sat Aug 18, 2012 at 07:07:06 AM PDT

  •  There's a high bar to (5+ / 0-)

    clear when an analyst says, "here's why I'm excluding only the two polls that make our guy look bad." I think the case is pretty solid for why to exclude Rasmussen. The Gallup calculus, on the other hand, is something like "well, we know their methods are solid, but something weird is happening." I'm not entirely sold on that -- though I readily acknowledge that something weird is happening. But again, that's why Pollster's charts are so great -- we can all pick our preferred combo.

    I'd love to see the raw, unweighted sample marginals for every single poll taken by every organization -- and then average them with localized regression, Kalman filtering, or whatever. With numbers that large, I'd trust sampling theory to average things out in the end, more than I'd trust the different weighting rules devised by pollsters.

    Grew a mullet and a mustache / Got a job at Chick-Fil-A

    by cardinal on Sat Aug 18, 2012 at 07:15:54 AM PDT

    •  this point exactly (3+ / 0-)
      Recommended by:
      cardinal, quill, pgm 01
      But again, that's why Pollster's charts are so great -- we can all pick our preferred combo.
      and mine:
      I understand that not everyone will agree, but the other feature of the polling graphs is that they are clearly marked as "customized", and you can easily add the trackers back in if you so desire. The action is transparent.
      Some notes:

      • ras has obama by 2 today. Who cares? Lousy methodology. I'm not using it.

      • polls should not be unquestioned. Now, I don't expect my objections to reach AAPOR submission-level, but I have them nonetheless (and they include frequency of polling whereby ras and gallup "flood the zone" with data compared to the once-a-monthers). And there is always the aggregate data without dropping anything (it's there at pollster or TPM or RCP). But I would be delighted if it made people think about the issue of blindly taking a poll at face value.

      • I am trying to be as transparent and consistent as possible. I am happy to chart the trackers and non-trackers separately and show everyone both graphs. Consistent use of a chosen baseline should be helpful. I am not excluding a given poll, i am electively excluding a pollster.

      • maybe gallup with publish an acceptable explanation. Happy to review it if they do. See Pew as an example.

      • as always, thanks for the thoughtful comments.

      "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

      by Greg Dworkin on Sat Aug 18, 2012 at 07:27:26 AM PDT

      [ Parent ]

      •  why not weight pollsters on past performance? (1+ / 0-)
        Recommended by:

        I believe Nate has been grading each pollster based on overall accuracy re final exit poll results. Why not use those scores to generate a weighting to apply to each pollster? That way you can't be accused of bias for discounting consistently bad pollsters like Ras. I don't know where Gallup falls on Nate's score, but if they've had an R bias through more than one cycle, then they should have a lower accuracy score despite being more "trustworthy".

        •  Nate, indeed does that (1+ / 0-)
          Recommended by:

          I don't know that I can do it competently.

          "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

          by Greg Dworkin on Sat Aug 18, 2012 at 09:14:38 AM PDT

          [ Parent ]

          •  shouldn't be too hard (1+ / 0-)
            Recommended by:

            In theory, as long as you generate and apply the weighting fairly, you can use any numerical score to generate a weighting. You don't even have to use Nate's data, or even historical performance. For instance, you could weight pollster tracking poll results based on (inverted) average deviation from the median of all recent tracking polls. That would punish all outliers, regardless of which party they favor.

            In practice those graph generators may not be sophisticated enough to handle custom weighting.

    •  on Gallup (2+ / 0-)
      Recommended by:
      DemFromCT, cardinal

      I basically agree with everything you said -- this verges on a quibble.

      The Gallup calculus, on the other hand, is something like "well, we know their methods are solid, but something weird is happening."
      Mark Blumenthal's analysis pretty much convinced me, provisionally, that Gallup's methods aren't entirely "solid."

      At the same time, I agree with quicklund that (in my own words) if we knew half as much about the other pollsters' methods as we know about Gallup's, we'd know that some of them are a lot worse than Gallup's.

      This discussion seems to be about choosing a least-worst ad hoc tweak in Huffpo Pollster World, given that Gallup's apparent house effect ends up having much more impact than any other individual pollster's. Simply filtering out Gallup definitely is not ideal.

      •  agree completely (2+ / 0-)
        Recommended by:
        HudsonValleyMark, cardinal

        and believe me, I am reluctant to filter anyone out. Slippery slope, that. I keep hearing Charles Franklin's voice in my head warning against cherry picking.

        But as long as the two separate curves posted look the way they look, I find it disturbing and hard not to at least look at them separately as well as together (standard model, which does not disappear).

        "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

        by Greg Dworkin on Sat Aug 18, 2012 at 11:08:30 AM PDT

        [ Parent ]

  •  Heh heh (0+ / 0-)

    You might want to include today's Rasmussen. It's Obama+2!

  •  I've said this before - (2+ / 0-)
    Recommended by:
    Mariken, Bush Bites

    I like leaving them in because it simulates the effects of election fraud and voter suppression.  It's correlation, not causation, but the actual votes will be closer than the non-Ras-Gallup composite.

    •  Wasn't it Stalin who said (0+ / 0-)

      It isn't the voting, it's the counting.

      Small varmints, if you will.

      by aztecraingod on Sat Aug 18, 2012 at 09:25:42 AM PDT

      [ Parent ]

    •  Simulation implies consistency. There is no (0+ / 0-)

      consistency to Ras.Their polls swing all over the place and are political all the way through. So the voter fraud idea just doesn't hold water. Honestly, I think a lot of folks here are so used to disappointment that Ras is an excuse to 'be prepared' for a loss. There is no rational basis at all for using Rasmussen. Zero.

      If I knew it was going to be that kind of party, I'd have stuck my ---- in the mashed potatoes! - Paul's Boutique

      by DoctorWho on Sat Aug 18, 2012 at 09:42:24 AM PDT

      [ Parent ]

  •  Thanks for this interesting explanation. (0+ / 0-)

    I would suggest adding a little annotation to each chart informing people that you have left out Rasmussen and Gallup and giving a link to this diary so that people can understand the reasons for leaving Gallup and Rasmussen out.

    The future is renewable.

    by KiB on Sat Aug 18, 2012 at 08:04:35 AM PDT

  •  I trust you on polling issues, Doc. (0+ / 0-)

    But I prefer worst case scenarios.

    BTW, just curious:

    Has Nate worked in Voter Suppression into his model?

    If Obama didn't get Bin Laden because he didn't pull the trigger; then Bin Laden didn't take down the World Trade Center because he didn't fly the planes.

    by Bush Bites on Sat Aug 18, 2012 at 08:05:34 AM PDT

    •  here (0+ / 0-)

      Measuring the Effects of Voter Identification Laws

      Starting with Saturday’s forecast, these shifts are applied to the “state fundamentals” calculation that the FiveThirtyEight forecast model uses along with the polls in each state. The state fundamentals calculation is based on large part on how the state voted in 2008 or 2004 — so if there have been changes in the fundamentals since that time which we think might have predictable effects, it is worth accounting for them.

      I do not apply any adjustment to the polls themselves. My notion is that the pollsters are responsible for accounting for these effects, such as by means of their likely voter models or their mechanism of accounting for registered voters.

      The effects of the adjustment are ultimately fairly minor. In Pennsylvania, for instance, it reduced Mr. Obama’s chances of winning the state to 82.6 percent from 84.2 percent, according to the model’s estimate. Still, it makes Pennsylvania a little closer, and slightly increases the chance that it will be the tipping point state in the election

      "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

      by Greg Dworkin on Sat Aug 18, 2012 at 08:48:42 AM PDT

      [ Parent ]

      •  I remember that article (1+ / 0-)
        Recommended by:

        but it only takes into account voter ID laws, and leaves out striking valid voters from the rolls, unequal access to voting facilities, etc. I know all of those factors are  changing all the time and we won't even know half the story til after the election, but it would be good to see conservative estimates of the effects of those factors on swing states modeled in.

    •  How does it prepare for a worst case scenario (0+ / 0-)

      if there is no consistency to the Rasmussen polls. That's the issue. It's not like 'house effect' internals - it's straight up dishonest polling. This is where Rasmussen succeeds on the Dem front, because so many people use it as their worry blanket when there is no reason at all for doing so. Gallup at least you can say 'house effect', etc. DemFromCT is on the money.

      If I knew it was going to be that kind of party, I'd have stuck my ---- in the mashed potatoes! - Paul's Boutique

      by DoctorWho on Sat Aug 18, 2012 at 09:47:42 AM PDT

      [ Parent ]

  •  methodology (0+ / 0-)

    The test of methodology is in the election results, not before. As has been noted, based on being outliers, PPP and Pew would be excluded before Gallup.

    The reason we know about the questionable aspects of Gallup is not because they are a poorer polling organization, but because they are one of the most transparent. The reason you don't know about the problems with other polls is that almost all of them are less transparent and don't give you enough information to look under the hood (Pew may be an exception here).

    You dismissed Nate Silver's polling average estimate which looks at performance in past elections as well as comparisons to other polls to come up with a much more sophisticated average because his model looks at economic and other materials. But that is only true of the Nov. 6th forecast. The "nowcast" excludes all that and only looks at current polling under his sophisticated averaging, including weighting for quality. And the nowcast shows 50.1% to 48.6%.

    Blumenthal may have some critiques for some aspects of Gallup, but I can't imagine that he would ever advocate removing them from a polling average. The folks at Gallup have a good reputation and a lot more expertise than you and they listen to and seriously respond to criticism, just like they do at Pew and they get divergent results. As Abramowitz acknowledges, in most aspects their polling techniques are first rate - much ahead of the average poll.

    The idea of excluding the poll where we know the most about its techniques, preferring the polls where we know the least of their techniques, is itself methodologically unsound, particularly if you don't personally have the educational background and experience of professional pollsters such as Blumenthal or the folks at Gallup and Pew. These criticisms haven't themselves been empirically tested against results and shown to cause a deviation from different adjustments. What we have is professional disagreement among qualified persons.

    Why do you feel comfortable adopting a technique based on a quality analysis that moves you further away from the most sophisticated average using a quality analysis?

    •  he doesn't advocate removing them (3+ / 0-)
      Recommended by:
      DoctorWho, madhaus, MadEye

      in fact, i know both co-founders of pollster fairly well, and Charles Franklin always advised "keep them all".  They are wise to do so, but mark did need to adjust this year for the frequency with which ras and gallup trackers come out in their algorithm.

      I'm not competing with pollster. I'm explaining what I'm doing personally, and why. I'm using their tool to demonstrate something not seen by adding them together.  Do what you will with it.

      For the record, i have not "dismissed" anything Nate does, I explained that the Nov 6 forecast isn't comparable (the nowcast is) because they look at different things.

      "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

      by Greg Dworkin on Sat Aug 18, 2012 at 08:54:11 AM PDT

      [ Parent ]

  •  Gallup Tracker has been unreliable (5+ / 0-)
    Recommended by:
    ekgrulez1, EcosseNJ, askew, Delilah, pgm 01

    Since it was first introduced in 1996 I believe.  The Clinton-Dole race was a Far more static race than the Gallup tracker led us to believe.  In 2000, they buried Gore; had him down double digits at times when Gore ended up winning the popular vote (and should've won the electoral college).  In 2004, initially a little too friendly to Kerry; later a little too sunny on Bush.

    In 2008, the tracker didn't really indicate whAt wAs happening at the state level.  That has been true for a while.

    Trackers are hypersensitive to changes in the news cycles, which is why they vary so much.  When you apply a restrictive voter screen, that also puts a ceiling on what one candidate Can do and provides more freedom of upward movement for the other.

    Ras may be O +2 today, but I'd rather take last month's view of Obama down in the trackers and up in Ras' polls of the various swing states.  

    What I want to see is a lot of swing state polling by different organizations and a lot of national polling.  That's how I knew with confidence that Obama was going to win in 2008, and knew that Kerry would need an inside straight to win in 2004.

    Alternative rock with something to say:

    by khyber900 on Sat Aug 18, 2012 at 08:13:32 AM PDT

  •  It would be interesting to see a PA poll with the (2+ / 0-)
    Recommended by:
    Bush Bites, Calamity Jean

    voter suppression models they are talking about thrown in.

  •  PS (1+ / 0-)
    Recommended by:

    One of the reasons I go on about this, is because in 2004 I was persuaded of similar criticisms of Gallup and by excluding them concluded that Kerry was even or ahead.

    IIRC, the RCP average showed Bush ahead by 1.5% and he actually carried the popular vote by 3%.

  •  Odd is not what the word I use for Ras. (1+ / 0-)
    Recommended by:

    Fraudulent is more apt.

    Today he would have me believe that Obama is +2 nationally (meaning a "bounce" of -4 for Ryan).

    OTOH, his State Polling has been trending Romney.

    I've been following Scotty's antics since before the 2000 Election and although his final numbers can be reasonably good, it's always appeared that he wants to create a narrative.

    It's much harder to dismiss Gallup other than their weighting causes them to have a house effect, but I'm reluctant to eliminate them because there are polls in there with a Democratic House Effect as well.  Just not tracking polls.  

    I simply do not trust Rasmussen to include him in anything other than guessing what he's trying to accomplish.

  •  Averaging polls is inherently a-scientific (3+ / 0-)
    Recommended by:
    exterris, Subterranean, pgm 01

    The better method is to overlay the various pollsters on the same graph, as you did on the Gallup/Rassmussen chart. Yes, it gets muddy and hard to read. But it does not mix apples, oranges, and grapes way averaging does.

    If several pollsters are seeing the same trends their curves will lay atop &/or parallel one another. Outlier pollsters will be obvious from their curve snaking through otherwise "unoccupied territory".

    But as for trends. The trend these days is to use this average-polls stuff. That's probably not going to stop.

  •  Rasmussen and Gallup cannot be statistically (0+ / 0-)

    valid.   To get that standard deviation, you would need nearly 4,000 respondents to get the 95% confidence level down to 2% on each poll, and that would presume that the selection was static.

    They should be called up on this plot and asked to explain how they can get +/-1% standard deviation and 2% 95% clustering in all those polls.

    Dem, you are really on to fraudulent polling here.

    The Muslim said "I wished I had met Christ before I met the Christians" - Rev. Marvin Winins

    by captainlaser on Sat Aug 18, 2012 at 08:55:05 AM PDT

    •  wouldn't go that far (0+ / 0-)

      this is a personal choice to see what the numbers look like with and without them.

      "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

      by Greg Dworkin on Sat Aug 18, 2012 at 09:02:27 AM PDT

      [ Parent ]

  •  Gallup (2+ / 0-)
    Recommended by:
    glorificus, IreGyre

    Thank you for this article.

    I watch the swing state polls very closely, but historically have not paid close attention to these "daily tracking" polls.

    For some reason, the other day I took note of the Gallup daily tracking poll and was astonished to see that Obama trailed by several points.

    Almost all the other polls I had seen showed Obama with at least a few point lead.

    This confirms my instinct that Gallup has a significant right lean. My instinct also tells me that this is not an accident.

    Gallup is and always has been the most recognizable brand in polling in this country. When people think of polls, the name Gallup is the first one that comes to mind. Even people who don't follow this stuff would know the name Gallup.

    I think Gallup is in the tank for the republicans. By putting out a poll that skews a certain way every day, you can subtly create a psychic narrative in somebody's head that isn't paying particularly close attention.

    Oh, Romney's probably going to win, I'll vote for him.

    I thought something was fishy before I read this diary, and now I'm almost positive.

    "Pay no attention to that man behind the curtain!"

    by jkay on Sat Aug 18, 2012 at 08:56:06 AM PDT

    •  i don't think they deliberately do it (0+ / 0-)

      I think it's a "modeling" issue, not an "intent" issue.

      "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

      by Greg Dworkin on Sat Aug 18, 2012 at 09:01:25 AM PDT

      [ Parent ]

      •  You are probably right (1+ / 0-)
        Recommended by:

        However, just because they are "transparent" in terms how they do a poll does not necessarily mean that they don't do certain things "intentionally" to obtain a certain result.

        If nine out of ten polls show Obama leading, and one for Romney, I find that to be "curious" and an indicator that something may be wrong with the methodology.

        What is 538's take on Gallup specifically?

        "Pay no attention to that man behind the curtain!"

        by jkay on Sat Aug 18, 2012 at 09:35:25 AM PDT

        [ Parent ]

      •  What about comment above that Gallup was sold (0+ / 0-)

        If a right-wing outfit bought the brand, wouldn't that explain it?  Same thing as News Corp buying the Wall Street Journal and The Times of London; grab the gold-plated names and serve crap on them, hoping most people will be swayed by their integrity being "golden."

        In capitalist America, bank robs you!

        by madhaus on Sat Aug 18, 2012 at 09:49:13 AM PDT

        [ Parent ]

        •  I just find it hard to swallow (0+ / 0-)

          Gallup has been the "gold standard" of polling forever and now they just happen to be the most right leaning poll, even more so than ras & fox.


          I can't refute the methodology because I'm not a statistician.

          As somebody who follows this stuff very closely, I have gotten a very good feel over the years for things that do not "smell right", and this is one of them.

          "Pay no attention to that man behind the curtain!"

          by jkay on Sat Aug 18, 2012 at 10:04:56 AM PDT

          [ Parent ]

  •  I was always annoyed TPM included (0+ / 0-)

    Rasmussen until I just started mentally adding 5-6 points to their results for the president.

    This is, of course, the difference between republicans and human beings. - Captain Frogbert

    by glorificus on Sat Aug 18, 2012 at 09:16:43 AM PDT

  •  Wouldn't it make sense (0+ / 0-)

    to compute, say, three sets of means on the same time axis: polls tending to lean R (dashed lines), polls tending to lean D (dotted lines), and other polls (solid lines)? That approach would turn historical bias into a way to get a more complete picture of the spectrum of polls.

  •  I think ... I didn't look it up ... (0+ / 0-)

    That the final Gallup tracker in 2008 had Obama by 11.  Maybe their GOP bias was on vacation that day.  In any event, it was pretty wildly wrong.

  •  The Prof. Abramowitz diaries should be rec'd (1+ / 0-)
    Recommended by:

    ...on sight. Easily some of the best writing on polls and polling methodologies you're likely to see on the net. The lack of comments and recs so far (he has written only a few diaries for Daily Kos so far) is disappointing.

    •  They should be (0+ / 0-)

      I was embarrassed that he was not on the rec list and had so few comments in comparison with some of the the other (os questionable quality).   He's brilliant and if he takes the time to post here, there should have been a more receptive response.  Thankfully Dem for CT referenced AA's diaries in his own pieces.      

      I vote Democratic because I am a woman with self-respect , who rejects bigotry of all kinds, subscribes to science, believes in universal health care, embraces unions, and endorses smart internationalist foreign policy.

      by Delilah on Sat Aug 18, 2012 at 12:30:56 PM PDT

      [ Parent ]

  •  as an aside, on 8/26 (0+ / 0-)

    1. aggregate without Rasmussen and Gallup:
    Obama 47.8, Romney 44.6

    2. Rasmussen and Gallup alone:
    Obama 46.0, Romney 46.2

    3. "All polls included" :
    Obama 46.4, Romney 45.5

    Both 1. and 3. [but not 2.] fit best today's NYT:

    With the race entering its final, decisive phase, strategists on both sides agree that Mr. Obama maintains a razor-thin edge.

    "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

    by Greg Dworkin on Sun Aug 26, 2012 at 05:03:02 AM PDT

Meteor Blades, Thumb, Alma, Timaeus, askew, abarefootboy, Lahdee, Emerson, Shockwave, cotterperson, hyperstation, eeff, zeroooo, JLFinch, Delilah, litho, sngmama, itskevin, revsue, Redfire, Texknight, tomephil, Chirons apprentice, FlyingToaster, zerelda, Jen Hayden, ExStr8, maybeeso in michigan, radarlady, kitchen sink think tank, llywrch, Dobber, Gary Norton, GreyHawk, Ice Blue, Savvy813, JanF, Jim R, BachFan, cybersaur, kestrel9000, smokeymonkey, Bush Bites, SadieSue, middleagedhousewife, BlueMississippi, doingbusinessas, Texdude50, kurious, Temmoku, cpresley, sephius1, pgm 01, dwakeman, Cronesense, camlbacker, FWIW, Msinformed, Mary Mike, joedemocrat, bnasley, mbh1023, US Blues, RokkiBlueboa, jnhobbs, gchaucer2, thankgodforairamerica, wayoutinthestix, prettyobvious, bythesea, elwior, skohayes, Sharon Wraight, pamelabrown, hwmnbn, smartdemmg, Gemina13, maggiejean, aufklaerer, bobatkinson, Remediator, petral, followyourbliss, elziax, oxfdblue, johnosahon, mdmslle, 57andFemale, haremoor, Tortmaster, astral66, Larsstephens, KroneckerD, womankind, gulfgal98, Puddytat, Floande, Actbriniel, tgrshark13, Onomastic, annieli, kerflooey, mama jo, kamrom, Quite Contrary, implicate order, EcosseNJ, cv lurking gf, OldGrammy, zukesgirl64, Cinnamon Rollover, awsdirector, MRA NY, Imhotepsings, Andrew F Cockburn, ratcityreprobate, thejoshuablog, Auriandra, ParkRanger, quill, Only Needs a Beat, KiB, We Won, IndieGuy, AreDeutz, Trotskyrepublican, JGibson, Mike RinRI, belinda ridgewood, MartyM, doroma, Near Miss, nomandates, countwebb

Subscribe or Donate to support Daily Kos.

Click here for the mobile view of the site