Skip to main content

I find myself checking DailyKos at least a dozen times (ok, maybe two dozen?!) a day for updated poll news. I scan past politicarewire and Nate's blog and RCP many, many times a day for the slightest hint of movement. At this point as soon as a new poll hits I've an awfully good idea (if not perfect) of how it fits into the context of the pollster's previous results, other recent polls in the state, its partisan lean, etc. My mood rises and falls on the barest hint of a trend one way or the other.

And really? It's a colossal waste of time.

My reptilian mid-brain can't stop being terrified. That fear that keeps me obsessively, obsessively checking: "Are we going to be ok? Will we just punt and continue to give even the hope for future change over to the Mourdock/Cheney/Romney bastards?"

My analytical brain knows that subtle poll shifts cannot allow us to deduce much in a quasi-close election. To extrapolate responses from any poll to what's going on in the general population requires really strong assumptions. The poll's "margin of error" is, most definitely, the least of it.

Partly as therapy, I want to describe some of the obvious complications with trying to "deduce" the mood of the population from a poll (any poll); the contents won't be news to anyone who's studied this stuff, but may well be to those who haven't. More below the jump.

Kindness of Nate Silver many of us have been schooled up in a large number of ways about the vagueries of polling.

We attend to sample size associated with a poll. Why? Because if the sample is too small then there's a greater likelihood that the sample isn't representative of the population.

We attend to which pollsters sample from landlines only, or also from cell phones. Why? If those who only use a cell phone are systematically, if only subtly, different from those who only or also have a landline then the sample won't be representative of the population as a whole; indeed, the parameters being estimated will be biased.

We attend to demographics of those polled. Why? Too many women, or men, or blacks, or latinos, or young, or old, or whatever and (again) the sample won't be representative of the population; the parameters being estimated will be biased.

We attend to whether the population being sampled from is of Registered Voters or so-called Likely Voters.

But it's only relatively recently that Nate clued me into the fact that poll response rates are incredibly low (of the few polls I've looked at half-way closely the response rate can be as low as 2%, and is rarely above 10%). Why does this matter if the sample that responds has all of the "appropriate" measurable features that you think are representative of the population as a whole (demographics, cell vs. land lines, etc.)?

It is not at all unreasonable to suspect that those who respond to a poll are "different" (representative of a different population from which the poll is a hopefully representative sample) than those who do not. After all, the very fact that so few people respond to polls is itself evidence that poll responders are exceptional people. The basic assumption that is generally made is that poll responders, while exceptional from the perspective of responding to the survey instrument, are nevertheless exactly typical (representative) of those who do not respond to the survey instrument in so far as voting preferences are concerned. That is, the population who does not respond to polling has the same preferences as those who do.  

This is a strong assumption. It would be super unsurprising if it's importantly wrong. And it would be unsurprising if what demarcates someone who is willing to respond to a survey compared to someone who is not so willing changes over time (so that predicting the "non response bias" based on past performance itself might be dodgy). After all, the intrusions of marketers and media and push polls and the like continues to grow. Those willing to engage in the process of polling, therefore, would seem to have thicker skins to that crap and such skins might (just might) correlate with voting preferences.

A simple math example is useful for understanding the degree to which this problem is serious. The probability that a random voter will vote for President Obama can be expressed as the following (where P( ) means "the probability of what's inside the brackets"):

P(will vote for Obama) =

   P(respond to the survey)*P(will vote for Obama, conditional on having responded) +
   P(not respond to the survey)*P(will vote for Obama, conditional on not having responded)

Let's assume that on average P(will respond to the survey) = 5%. Then:

P(will vote for Obama) =
     5%*P(will vote for Obama, conditional on having responded) +
     95%*P(will vote for Obama, conditional on not having responded).

The survey instrument measures P(will vote for Obama, conditional on having responded). Accordingly, the thing we really care about -- P(will vote for Obama) -- is determined by the 95% weight on something we never observe, and cannot observe. That unobservable probability needn't be very different, at all, from the observable probability (the sample that responds to the survey) for the polling instrument to be importantly at odds from the actual population's intentions of how they intend to vote.

I'm not saying that polls tell us nothing. What they can tell us, perhaps, is about trends in sentiment. But what keeps me up at night, what makes me unable to breath, what fills my heart with fear and anxiety are not trends, but levels, of voter sentiment. With all my heart, for me, and us, and especially my kids I want P(will vote for Obama) > P(will vote for Romney). And the real bitch? I can't know this from polls. I can more or less reasonable inferences. But given that the really crucial thing that matters isn't observable, I cannot deduce what this is.

Originally posted to AlaskanAnt on Fri Oct 26, 2012 at 05:05 PM PDT.

Also republished by Community Spotlight.

Your Email has been sent.
You must add at least one tag to this diary before publishing it.

Add keywords that describe this diary. Separate multiple keywords with commas.
Tagging tips - Search For Tags - Browse For Tags


More Tagging tips:

A tag is a way to search for this diary. If someone is searching for "Barack Obama," is this a diary they'd be trying to find?

Use a person's full name, without any title. Senator Obama may become President Obama, and Michelle Obama might run for office.

If your diary covers an election or elected official, use election tags, which are generally the state abbreviation followed by the office. CA-01 is the first district House seat. CA-Sen covers both senate races. NY-GOV covers the New York governor's race.

Tags do not compound: that is, "education reform" is a completely different tag from "education". A tag like "reform" alone is probably not meaningful.

Consider if one or more of these tags fits your diary: Civil Rights, Community, Congress, Culture, Economy, Education, Elections, Energy, Environment, Health Care, International, Labor, Law, Media, Meta, National Security, Science, Transportation, or White House. If your diary is specific to a state, consider adding the state (California, Texas, etc). Keep in mind, though, that there are many wonderful and important diaries that don't fit in any of these tags. Don't worry if yours doesn't.

You can add a private note to this diary when hotlisting it:
Are you sure you want to remove this diary from your hotlist?
Are you sure you want to remove your recommendation? You can only recommend a diary once, so you will not be able to re-recommend it afterwards.
Rescue this diary, and add a note:
Are you sure you want to remove this diary from Rescue?
Choose where to republish this diary. The diary will be added to the queue for that group. Publish it from the queue to make it appear.

You must be a member of a group to use this feature.

Add a quick update to your diary without changing the diary itself:
Are you sure you want to remove this diary?
(The diary will be removed from the site and returned to your drafts for further editing.)
(The diary will be removed.)
Are you sure you want to save these changes to the published diary?

Comment Preferences

  •  I'm the same way... so I guess that makes us (24+ / 0-)

    Obsessive ComPOLLsive!

    drum roll

    I am the neo-con nightmare, I am a liberal with the facts.

    by bhfrik on Fri Oct 26, 2012 at 05:14:58 PM PDT

  •  I share your sentiment... (7+ / 0-)

    but for what it's worth, we are starting to get some information.  The early voting numbers, for instance can (at least in principle) be checked against what pollsters say are the numbers and party IDs of early voters.  My impression is that they aren't wildly off-the-mark (though admittedly, I would love a more thorough analysis).

    But there's also the fact that polling isn't entirely new.  You'd be right to have these fears in almost every election cycle, but for the most part, we've had a pretty good idea of who was going to win beforehand.

    As much hand-wringing as we did in 2008, we were all pretty certain that Obama had it in the bag, and people like Nate Silver were basically able to call not only the race, but nearly every state in the race using polling info as input.  

    While 2004 was heartbreaking, the race really looked neck and neck (from an electoral standpoint) throughout.  This one doesn't.  Obama has Ohio.  He has Nevada.  He has Iowa.  He probably has New Hampshire, Colorado and Virginia, though at this point, it's just gravy.  Point is that Romney hasn't been ahead at any point in this election cycle.  While the margins are small, they aren't statistically insignificant.  

    And remember, it's the other side that keeps insisting that we can't learn anything from the scientific method.

    •  Thanks for the encouraging words, kind doctor... (7+ / 0-)

      but the information that we do get, or from which we hope to gain comfort, is nevertheless plagued by the very issues I identify.

      This is not to suggest, for an instant, that I don't appreciate your kind and encouraging words. I do. The point of my post is that what you express is really, at day's end, a kind of faith. Communicating that faith to me is lovely, and supportive. But for some reason I'm caught in this stupid loop where fear wants only to be assuaged by scientific certainty.

      I left my office this evening to walk down to my car to drive home. The sun was setting gorgeous orange and pink over Cook Inlet, the range snow covered and reflecting the glory of the day's end back to me. If THAT can't be retort to what ails me I don't know what can. This election matters, it's deadly serious. And the glory and beauty of the world is right here. Right now. Breath!

      •  Oh, I miss Alaska! (4+ / 0-)

        I used to get up there frequently, but don't anymore. Please take an extra walk or two now and then, and drink up the cool air and the extraordinary views for me.

        What a blessing Alaska is. Yes, it should fix up what ails you. If not, know that you have friends here who understand.

        But I'm also going to  urge you to GOTV as well. For 10 days, you can try to ignore the reporting of the polls and try to actively change them some.

        © grover

        So if you get hit by a bus tonight, would you be satisfied with how you spent today, your last day on earth? Live like tomorrow is never guaranteed, because it's not. -- Me.

        by grover on Sat Oct 27, 2012 at 02:05:03 AM PDT

        [ Parent ]

  •  We're truly at the end of this campaign now (12+ / 0-)

    I think it's supposed to be like this, unfortunately. Everybody out there wants to add something new to the story to throw us off.

    Please know you aren't alone, which is why you and I and all of us come here hourly to see what's going on in the election.

    Markos had one message today that was pretty loud and clear - GOTV Period.

    It's all we have left to do. And at this point, it's better to keep busy doing anything campaign related.

    When it's over and President Obama hopefully gets re-elected, we can all sit back and be proud that we were a part of his two campaigns for the presidency.

    And that we did everything we could to make it happen.  Don't forget, we're not just trying to beat Romney. We're trying to beat a huge right wing power grab.

    We just might beat them. Won't that be something? Us little 'ol democrats with very little media may do what I thought was once almost impossible.

    Beat the bastards.

    •  Indeed... (7+ / 0-)

      Your closing salvo brings to mind something that Edward Abby used to say. (I miss that guy!)

      "One final paragraph of advice: do not burn yourselves out. Be as I am — a reluctant enthusiast... a part-time crusader, a half-hearted fanatic. Save the other half of yourselves and your lives for pleasure and adventure. It is not enough to fight for the land; it is even more important to enjoy it. While you can. While it’s still here. So get out there and hunt and fish and mess around with your friends, ramble out yonder and explore the forests, climb the mountains, bag the peaks, run the rivers, breathe deep of that yet sweet and lucid air, sit quietly for a while and contemplate the precious stillness, the lovely, mysterious, and awesome space. Enjoy yourselves, keep your brain in your head and your head firmly attached to the body, the body active and alive, and I promise you this much; I promise you this one sweet victory over our enemies, over those desk-bound men and women with their hearts in a safe deposit box, and their eyes hypnotized by desk calculators. I promise you this; You will outlive the bastards."

      I think I need more Ed-Zen, and I'll get through. Maybe time for a run.

      •  The extraordinary thing is that OFA (6+ / 0-)

        has made it possible to GOTV from an office in our home states or even in our homes if we prefer to not be one of of those "boots on the ground" people.

        Not everyone has to be in Ohio or Colorado. There's a sign in the office that I volunteer at that says 180 calls = 1 vote. That's what the research says.

        We each call. Together, we gather votes.
        I'm volunteering in ways that make sense for me. I volunteer for OFA and for our local Congressional Dem challenger. And I'm careful to draw boundaries so that it doesn't consume my entire life. I've done that before, and it's kind of insane; at least it is for me. By working some for both campaigns it's easier for me to say "no."

        As Ed Abbey said, Don't burn yourself out. 2014 is around the corner. As 2010 proved, off-year elections are incredibly important.

        So I dive in 100% and do what's most needed when I'm volunteering. I turn off the tv and computer much of the time when I'm home. I engage in hobbies I love and go outside to remember what I'm fighting for whenever I can.

        I spend time with loved ones, and not in front of the tv. MSNBC is just one big loop of the same news. It CAN be turned off.  

        And the polls are like a pot of boiling water. They don't need me watching them to change. They'll be there in 3 days. In fact, the more all of us work, the better they'll likely be.


        Balance. It's all important.

        © grover

        So if you get hit by a bus tonight, would you be satisfied with how you spent today, your last day on earth? Live like tomorrow is never guaranteed, because it's not. -- Me.

        by grover on Sat Oct 27, 2012 at 01:58:01 AM PDT

        [ Parent ]

  •  I am the same ... (4+ / 0-)
    Recommended by:
    blueoasis, ladybug53, rubyr, AuroraDawn

    ... I look daily. I look multiple times a day. I realize a poll is but a snap shot of what people were thinking at that moment. I realize that it must be a representative sample to be accurate. I ponder who the hell these undecided voters are when there is such a clear difference between candidate and parties. I wonder what could possibly sway them now that hasn't been said already. What do the dailies say? How's the EC? I honestly think I won't have a restful night until after the election. So I'm with you AlaskanAnt. All the way from the Bay State time zones away. GOTV. Engage on FB. Keep reading. Keep educating the low information voter. Peace.

  •  exactly (4+ / 0-)
    Recommended by:
    ladybug53, finley, rubyr, AuroraDawn

    Who can sleep.... I'm terrified... I had to finally subscribe to NYT digitally to check out Nate, because I used up the 10 free ones on every lap top, desktop and cell phone in my house. (I have 5 teenagers and an unobsessive husband if that tells you anything) and so is every other democrat and some sensible Republicans (Older, who just cant change their party ID and are hoping for a moderate take over) I know. I have to think... this fear must be motivating...maybe more motivating than Hope and Change? Keep Calm and GOTV

  •  Might I suggest visiting Daily Kos Elections? (0+ / 0-)

    You'd probably enjoy it.

  •  Seeing Romney's numbers rise (1+ / 0-)
    Recommended by:

    Seeing Romney's numbers rise toward 50 is making my life miserable.  What the hell do people see in him?  It can't be just media fluffing.  Is the Obama hate that strong?  That widespread?

    I am happy with some of the swing state numbers, especially my beloved home state of Ohio. But Wisconsin . . . I don't know if I can ever trust that state again.

    •  Frankly, I don't think Romney's... (1+ / 0-)
      Recommended by:
      Be Skeptical

      ratings have anything to do with Romney. I've met very few people - including some hardcore Republican colleagues and relatives - who actually like, let alone trust, Mitt Romney.

      As one of my cousins informed me - "I'm not voting for Romney. I'm voting against Obama".

      I tried to explain the man's background to him, and it made no difference. He would vote for anyone the GOP nominated to run against Pres. Obama. Given the amount of bigotry, fear, and hatred that is emanating from the right-wing, I think the GOP could have placed a ficus plant on the ballot to run against the President, and right-wingers would still have turned out to support the plant rather than vote for Pres. Obama. There is nothing rational about it. They just can't stand the President.

      Not even I expected Romney to let his entitled Lord-of-the-Manor freak flag fly as proudly as he did on Tuesday night ~ Charlie Pierce

      by AuroraDawn on Sat Oct 27, 2012 at 05:39:29 PM PDT

      [ Parent ]

  •  Honestly, the "snapshot" criticism of (0+ / 0-)

    polling is much better than sampling/screen/turnout criticism.  

    When you're at that level of mathematical modeling, I would question any license to any assumption except for maybe the most small "c" conservative assumption (i.e. that this presidential would resemble the last).  As a result, I think the polling world is really being used in the wrong way.  Polling would show you what would happen today under various assumptions of turnout.

    Since a) you don't know who turns out until they turnout and b) the election isn't held on the day in question, I largely find sampling/screen/turnout to be as silly when D's do it as when R's do it.

    I'll always be...King of Bain...I'll always be...King of Bain

    by AZphilosopher on Sat Oct 27, 2012 at 04:09:01 AM PDT

  •  I'm in favor of Universal, Single Payer Polling. (1+ / 0-)
    Recommended by:

    That which polls every single person every single time. It should also have a mandatory participation requirement to eliminate that pesky "sample" crap. Perhaps some kind of monetary penalty for not answering all the questions in a forthright and non-sarcastic manner.

    Let's go back to E Pluribus Unum

    by hazzcon on Sat Oct 27, 2012 at 05:06:19 AM PDT

  •  I'm just waiting for it to be over. (3+ / 0-)
    Recommended by:
    wishingwell, MillieNeon, AuroraDawn

    Then hopefully all those one-line "diaries" about polls that would barely make a respectable comment or a meager tweet will go away.

    Fuck me, it's a leprechaun.

    by MBNYC on Sat Oct 27, 2012 at 06:03:39 AM PDT

  •  I often check the Princeton Election Consortium (2+ / 0-)
    Recommended by:
    auditor, AuroraDawn

    and have learned that it changes its Electoral College numbers 3 or 4 times a day.  I'm pleased this morning to report this:

        As of October 27, 8:01AM EDT:
        Obama: 297
        Romney: 241  
        Probability of Obama re-election: 89% to 97%

    The spirit of liberty is the spirit which is not too sure that it is right. -- Judge Learned Hand, May 21, 1944

    by ybruti on Sat Oct 27, 2012 at 06:24:25 AM PDT

    •  It's funny, isn't it? (2+ / 0-)
      Recommended by:
      ybruti, AuroraDawn

      Small changes, small changes, obsessively remarked upon.

      "It looks like Ohio's firming -- Good news!".... "And hell, they'll probably steal Ohio anyway..."... "Maybe Virginia really CAN go for Obama -- Good news!".... "Oh crap, Florida for Romney?! Really?! Can that be true? Oh noooo!".... "And those national polls -- CRAP! If they're right the state polls aren't... "

      There's this endless do-loop in my head of doubt and hope. I don't need to feed it, I need to lobotomize it! Because really? The point of my original post is that we really cannot KNOW the state of the race and obsessing over what might be hints of electorate mood swings does not serve. I'm just not confident in assuming that non-responders have the same preferences as poll responders. Nate Silver has an excellent relatively recent post that compares national poll predictions, by firm, to outcomes. What is striking is the degree to which folks do not "get it right". I'm thinking that the biggest problem isn't sampling methodology, but the epistemic inability to measure P(will vote for Obama, conditional on not responding to the poll).

      •  Yes, now the 3 pm numbers from the Princeton (1+ / 0-)
        Recommended by:

        Consortium are less rosy, but still very good:

            As of October 27, 3:01PM EDT:
            Obama: 290
            Romney: 248
            Probability of Obama re-election: 85% to 95%

        The spirit of liberty is the spirit which is not too sure that it is right. -- Judge Learned Hand, May 21, 1944

        by ybruti on Sat Oct 27, 2012 at 01:13:59 PM PDT

        [ Parent ]

  •  I think the elections of 2000 and 2004 play into (1+ / 0-)

    this checking of the polls frequently too as we know the pain of those elections, they are in the not so distant past.  It is in the back of our minds all of the time. There is a little bit of Election PTSD because of those 2 elections,, I have believed that for a while.

    Follow PA Keystone Liberals on Twitter: @KeystoneLibs

    by wishingwell on Sat Oct 27, 2012 at 07:35:41 AM PDT

  •  Greg Abbott , Attorney General of Texas (1+ / 0-)
    Recommended by:

    Do not speak for me as a Texan, and may have violated Texas Official Oppressiopn laws by threaten to arrest poll watchers ,maybe the international obervers could file a complaint with the Travis County district Attorney ,that could bring Abbott before a Austin grand jury to explain his action

  •  My fairly worthless two cents (1+ / 0-)
    Recommended by:
    Be Skeptical

    This whole "we're barely winning even though everyone else says we're not" seems to be confirmation bias. Our leads are very tenuous in an unfavorable environment.

    Willard's huge debate bounce should tell you all you need to know about that. Joe Q. Public gets their first look at the guy and automatically he's viable. There's a reason for that: this is not a favorable environment for us. I think the average, non-partisan voter feels like it's not all that bad, but it's not all that great.

    I am glad the media is pushing the too close to call horserace. The only poll that matters is the polls where the votes are cast. In fact, I hope that they continue it because the possibility of losing will energize our ground game, where we have an advantage. I believe this is the sole reason the President has gone on his redeye extravaganza tour of the swing states much earlier than usual for a candidate. It is not as much about persuading pesky undecided voters as it is for us to get our asses out there for him. He's telling us "look, I'm working my ass off out here, I can barely speak, so work your ass off, too."

    When I hear about all these small leads we have, I think of Florida in 2004 where Kerry seemed to have a small lead toward the end. He lost Florida by 5 points. Bush closed in Wisconsin strongly in 2004. He lost the state.

    Colorado, Florida, Virginia, and North Carolina are particularly worrisome for us. I am also not sold on New Hampshire. I do think we have a real lead in Nevada and Iowa.

    Once again, I think history will write that Ohio is the deciding state. Republicans are going to go to the polls in droves to vote against Obama. Count on it.

    Our "firewall" consists of Ohio. Ultimately, history will note Romney's puzzling obsession with bringing up the auto bailout will be what barely pushed us over the edge to victory, but we have to make that happen on the ground.

  •  Have you seen this one? (0+ / 0-)

    Rand: American Life Panel.

    Different methodology, different sample size.  Very interesting approach that actually show swings within the same group of voters.

    Only a national poll though.  IMHO I think things will open up for Obama and he will win.  Maybe there will even be a little surprise on the evening of the 6th.

    I follow the polls but not compulsively (then again I could be in denial).

  •  You are suffering from Electoral OCD... (0+ / 0-)

    I, too, am afflicted with this ailment. Thankfully, just like Romnesia sufferers, Obamacare has us covered!

    I can't wait for this Election to end. The waiting is so nerve-racking.

    Not even I expected Romney to let his entitled Lord-of-the-Manor freak flag fly as proudly as he did on Tuesday night ~ Charlie Pierce

    by AuroraDawn on Sat Oct 27, 2012 at 05:26:15 PM PDT

  •  October surprise...Hurricane Sandy... (0+ / 0-)

    I'll be watching the weather channel rather than the polls. If the weather is not as horrible as some predicted due to Hurricane Sandy, Obama wins, period.

    If some swing States get really ugly weather, it could lessen voter turnout.

    That would nailed Ohio and perhaps Virginia for Obama because he has such a huge lead in early voting.

    But PA does not have early voting. You have to vote on that day. Philadelphia has a huge Obama following.

    Can Obama still win without PA?

    •  I too, am transfixed with the polls, and then the (0+ / 0-)

      gut-fear that really makes me puke is that even if the polls put a win in the O column, the Swift Boating Rethugs are pros at "disqualifying" voters with challenges only solved at court level, hanging chads, slowing down counts, lengthening lines, having the media report the voting times are intolerable- why bother? etc, and then the Dems just go, shucks we lost (another) one- and we are plunged into an oblivion that will make the Reagan error pale-pasty-white in comparison.
      My twenty year old son says to me, most earnestly, that people are not so gullible or stupid to actually vote for a complete "tool" like Romney and I have to remind him of the unthinking grin and Texan oil bidness man who was put in office twice- someone actually voted for him (and he was our guvnor many years before that). That idiot grin on W and the malevolent sneer of Cheney coupled with the sheep-like acceptance (adoration?) of so many proud, card-carrying Republicans as they circled the economic drain of GOP origin, all the while pronouncing what a fine, fine President he was. No, race has nothing to do with it, why do you ask?
      Oh, the thrill of having a president who can actually answer a question intelligently and say something  that is thoughtful and unrehearsed.
      Obama ain't perfect and he has often been equal parts thrill and chill, but imagining the crash and burn world we would have if he were somehow to lose re-election- the social sharks in chum filled waters strewn with the bodies of the non-one percent.  

      We have to hit the streets loudly when the chads start hanging or the Justices start deciding when enough votes have been counted- to allow the culmination of the Reagan Revolution would prove socially fatal to the nation as we know it. There aren't enough bridges available for us all to live under.

      Thank jebus for Nate Silver's reassuringly accurate polls of hope to help me sleep at night.
      Rant complete.
      Just sayin'.


      Occupy- Your Mind. - No better friend, no worse enemy. -8.75, -6.21<> Bring the Troops Home Yesterday

      by Thousandwatts on Sat Oct 27, 2012 at 10:59:50 PM PDT

      [ Parent ]

  •  I don't believe in polls unless the polling result (0+ / 0-)

    represents only the people polled.  I don't care what mathematical equation people use to explain the results, no 1,000 or 10,000 people polled can determine who 200 million voters will vote for.  I trust polls as much as I trust Nielsen and   a lie detector machine.  Both are hogwash.

Subscribe or Donate to support Daily Kos.

Click here for the mobile view of the site