Skip to main content

View Diary: Two Focus Groups (On Barack's Acceptance Speech and Sarah Palin) Updated (279 comments)

Comment Preferences

  •  Luntz also did a focus group on Obama Vs McCain (108+ / 0-)

    of a group of eldely undecideds sponsored by AARP, and Obama won 15 to 10 in this older group, after seeing the speech Thurs.

    The fox reporters were so mad. They couldn't understand how Obama could be seen as more accountable, caring, and experienced, detailed than McCain or Palin.

    They absolutely hated Palin!  

      •  Keeping watching CSpan. They may replay it. (7+ / 0-)
        •  A Few Interesting Points on the Methodology Used. (9+ / 0-)

          Sorry for Jumping up to the top.  I made a comment below that I think people will find helpful/informative regarding the Methodology/Technology used bu Luntz.

          Nice work Jen

          A few things on your analysis though... (0 / 0)
          And the only thing that made a good impression was the assertion that she fought the "Bridge to Nowhere" which was false, we know now.
          The two demographic segments that were represented on the screen of the Palin Focus Group were "Democrats" and "Republicans."  Probably leaners.

          I didn't catch the whole session yet, but from the few minutes I watched:

             -The Republicans reacted favorably to her "Gas Pipeline" comment

             -Both Democrats and Republicans reacted favorably to her "Stop wasteful spending" and "Cut property tax"

          This methodology allows for the analysis of many different demographic groups.  The participants would heve all answered a questionairre through their handsets prior to the viewing and the discussion.

          The "tracers" that we could see represented only two of those demographics.  The software would have also picked up Gender, Race, Income Level, Most Likely to vote for, etc. (use your imagination as to what else they could be categorized as - soccer mom, NASCAR dad, hockey mom, etc.).

          With Luntz at the helm, we know it is a Republican financed focus group - yes, AARP is involved, but guarantee that there are other Luntz clients as well).

          So, with these "People Meter" discussions one must be very careful in how they interpret that data.

          The tracers we saw were only the Democrats and the Republicans.  There are many other "subsets" that the technology would have been picking up on, that we never saw.

          I guarantee you that the "backroom subsets" (Soccer Moms for example) are the ones that Luntz' clients are watching most closely.

          From those categories that they create behid the scenes, comes much of the strategy.

          Plus, the sample size of 25? is very small.  It would provide a very significant margin of error.  But, many ideas will come out of it, and those ideas will be tested more in the near future.

          Rove digs deep with this

          McCain - Voted Worse Than Bush in 2000

          by Mr Magu on Mon Sep 01, 2008 at 10:01:19 AM PDT

          [ Parent ]

          •  Yes (3+ / 0-)
            Recommended by:
            Delilah, ZinZen, oobi

            It would provide a very significant margin of error.

            And let me underline that.  Focus groups, like other kinds of product research (usability studies, play testing ,etc) aren't interested as much in being statistically valid.  The goal ins't to quantitatively say "98.42% with a P value of 0.05 of all people liked the oil comment" the goal is to say "Some of the participants said they liked the oil comment because it said she was $blah$".  Going the whole double blind, statistically valid route is actually a waste of money because you really dont care much about accuracy... you just want to find broad themes and issues that need to be addressed in your product (or candidate).

            So just keep this in mind.  Lots of scentified/educated people unfamiliar with the subject often dismiss focus groups/product research because it isn't done as some double blind "scientific test".  Most of them become convinced that focus groups/usbaility studys are a good thing after they've watched videos of them being done on their products though :-)

            I will say though, it is important to do a good job screening your participants.  You dont want shy people afraid to talk and in most cases you dont want somebody who is in expert in market research (because they will be busy analysing your methods :-)  

            If you get the wrong set of people in your focus group, your results can be worthless.

            •  Results are critical (2+ / 0-)
              Recommended by:
              sharman, spookthesunset

              If you get the wrong set of people in your focus group, your results can be worthless.

              Because so much is riding on all of this analysis, some of these focus group researchers who implement sociometric methods & techniques must've certainly studied JL Moreno thoroughly and his implementation of actors/moles/agents to help facilitate positive outcomes?

              The question is, which focus group researchers fall prey to this?

              Frank Luntz's use of linguistic priming is on display here, as well. Example- one of his questions to the group:

              "How many here think Sarah Palin's inexperience in more important than Barack Obama's inexperience?"

              Before Jesus, there was love. Without love, there would be no Jesus.

              by oobi on Mon Sep 01, 2008 at 11:01:14 AM PDT

              [ Parent ]

              •  asdf (2+ / 0-)
                Recommended by:
                sharman, oobi

                "How many here think the lack of horse power of the Honda Prius is more important then the lack of horsepower of the Honda Civic?"

                This is a two part question.

                First, is this a sham and not really a focus group.  It would be a waste of money to try to bias your focus group when the whole point is to tease out both good things and bad things about your product.  Why the hell would you even pay for one if you are gonna bias it so much you dont learn anything about your product?

                Second, if this is a legit focus group (and I belive it is) is Luntz a good facilitator?

                Given we dont enough about what kind of product the AARP is working on that requires this focus group, it is hard to answer #2.  I personally think it is legit but too many people mis understand what focus groups are about.  They are not about "ramble on about how great Obama is but try to bias everybody to McCain".  They aren't even about staying neutral... if you were Microsoft and wanted to know why everybody likes google so much, you'd probably ask "What is so bad about Microsoft's search results?"

                PS:  I love market research and I wish more engineers and managers would learn to see the value in it.

                •  acknowledged (1+ / 0-)
                  Recommended by:

                  PS:  I love market research and I wish more engineers and managers would learn to see the value in it.

                  I think that they are. They just don't need to gather people as much, anymore.

                  Now with the myriad of datamining possibilities, they can market research and network analyze from the comfort of their war rooms.

                  ipod most played>>speech-to-text voicemails>>IP address logging at librul websites>>comments, password preferences, what you ate for lunch on your AMEX card>>GPS location of you and your top 5 during the convention>>and so on, and so on....

                  Before Jesus, there was love. Without love, there would be no Jesus.

                  by oobi on Mon Sep 01, 2008 at 11:36:50 AM PDT

                  [ Parent ]

                  •  Datamining is useless (2+ / 0-)
                    Recommended by:
                    mmacdDE, oobi

                    Because data mining doesn't allow for randomness.  When you data mine, there is a strong tendancy to get the results you want to hear.  Same with surveys... people (usually with no market research education) write the surveys in a way that tell them exactly what they want to hear.

                    A survey might ask:
                    "Was it easy to set the clock on the VCR" - "Yes/No"

                    That question never allows for the possibly of the "correct" answer which is "The VCR should set itself!"

                    Hooking some Linux distro with wires to phone home and say "How many users were able to change the screen resolution to work with their monitor" might sound like a good idea to an engineer, but it never allows for the possibility that a user should never need to change the resolution at all!

                    There is no substitute for having real people look at your product and answer your questions.  You'll never get random answers like "polar bears" when you data mine because.  You'll never get "I like Obama, and I think he is more accountable" if you data mine.  Data mining and surveys run high risks of setting yourself up for a giant feedback loop because you'll only get answers to the questions you think of asking.  The best, most useful answers are those to questions you never thought to ask in the first place!

            •  Probably the most important aspect of this (1+ / 0-)
              Recommended by:

              methodology is the makeup of the group.

              A good moderator is good too.  For what Luntz does, he is good at pulling information out of all kinds of voters.

              McCain - Voted Worse Than Bush in 2000

              by Mr Magu on Mon Sep 01, 2008 at 12:05:28 PM PDT

              [ Parent ]

          •  Any idea how these people were recruited? (1+ / 0-)
            Recommended by:

            I've worked with focus groups and it was always very hard to get a good psychographic to attend these things. People would usually agree for the $50-$75 and free food. So who are we really listening to here? I saw only a few minutes of this and was not encouraged. Did I miss something?

            •  No idea (0+ / 0-)

              But round here in Seattle, the average hour long focus group is $100 -> $150 cash.

              Most phone screeners usually start with "Do you work in market research, advertising, blah blah", ask if you or anybody you know work in the industry, if it is a mock jury trial ask if you know any of the people involved in the case.  Then they'll ask things like age and income bracket.  Ask you some question that has no right answer to judge if you aren't shy and would say something useful.  They'll toss in some questions about various products in the same product category.

              I wonder if these were AARP members (probably)--this also fits them into an age bracket.  I wonder if they were screened for how long they've been a member.  I wonder if they were asked something like "how much did you follow the primary" to gauge their interested in the election--and if they asked if they were interested in those who followed it, or those who did not.

              Who knows...  there were probably different groups that were pulled in that we didn't see.

            •  It depends. (0+ / 0-)

              Random off the street recruiting.

              Recruited by phone, internet.

              Payment varies significantly -- free tix, free stuff, money.

              McCain - Voted Worse Than Bush in 2000

              by Mr Magu on Mon Sep 01, 2008 at 12:11:17 PM PDT

              [ Parent ]

        •  I saw a part of the AARP focus on CSpan (5+ / 0-)
          Recommended by:
          javelina, sherlyle, EquiStar, bfbenn, Amber6541

          The one thing I disliked the most about the whole thing was how the coordinator kept trying to pigeon hole people on any particular issue.  "experience vs accountability vs judgment"  We all know that you will never find one person who agrees with any candidate on every issue.  It's always a blend of a large scale of topics.  I don't agree with Obama on a number of issues, but he's most definitely closer to me than McSame can ever be.

          We are not princes of the earth, we are descendants of worms. Nobility must be earned. - PZ Myers

          by Mercuriousss on Mon Sep 01, 2008 at 10:07:29 AM PDT

          [ Parent ]

          •  That is the point though (1+ / 0-)
            Recommended by:

            The AARP or whomever needs to know what people mean when they say accountability.  Imagine doing a play test where all the participants said was "the game should be fun!".  You have to keep prodding them to tell you what they mean by "fun".  The participants saying "the person needs to be accountable" is just as helpful to improving the "product" as somebody saying "the phone needs to be easy to use" or "the game should be fun".  What do you mean when you say "the candidate should be accountable"?

          •  I noticed that too (1+ / 0-)
            Recommended by:

            And I can't help but wondering - where do they find these people?
            I don't mean to sound "elitist", but what would motivate people to participate in one of these focus groups?
            I've always thought that basing strategy on these groups is an iffy proposition at best.  Tactics maybe, but your strategy?  Yet that seems to be exactly what McCain's campaign is doing.
            I'm sure I'm not the only one who hangs up on ANY robocall, and don't give much more respect to live callers of this ilk.  I can't imagine what would motivate me to participate in a "focus group" for something this important, much less what spokesperson or box design I prefer for that tasty new cereal or bottled water product.
            I'm absolutely convinced that this dynamic has as much to do with skewing poll results as does the lack of a land line with some demographic segments.  I know I've seen a diary or two indicating that the percentage having a cell as their only phone does not skew the results, but how about the percentage of people that "opt out" of the whole polling/marketing paradigm?  The combination of these things just HAS to mean something in terms of the accuracy of the results, doesn't it?  How many people will they actually contact to get a survey size of 1,000 people?
            Unfortunately, could it be that what DOES reinforce the accuracy of polls is the reporting or interpretation?  Does the prediction of the poll reinforce the conventional wisdom such that the prediction of the poll proves to be accurate?

            As you say, there is NO perfect candidate, there's always something to disagree with - I try to consider that when making a decision.  I really thought that's what you're supposed to do!

            Geez, that looks pretentious, but maybe someone gets my point?  The time for tactics is a few days before the election.  This is a period for implementing a longer term strategy, and all we ever get from McCain are these publicity stunts.  The Obama campaign seems to get that whole strategery thing...

            "The Attorney General underwent surgery to remove a faulty gall bladder...A condition that left his gall completely unmitigated" -Jon Stewart

            by bfbenn on Mon Sep 01, 2008 at 11:09:59 AM PDT

            [ Parent ]

            •  how it is done (2+ / 0-)
              Recommended by:
              oobi, Mercuriousss

              but what would motivate people to participate in one of these focus groups?

              $75->$200 cash, green dollar bills, depending on how long the focus group is.  You show up, they give you pop and free food and you leave with a wad of crisp twenty dollar bills.

              where do they find these people?

              There are companies who specialize in recruiting people for focus groups/usability studies.  You hand them a script (the screener) and they fetch you however many people you need for your study.  You typically pay the recruiting company based on how many people show up.

              I can't imagine what would motivate me to participate in a "focus group" for something this important

              No offense, but they screen specifically for your "type".  And by "type" I mean somebody with especially strong opinions on a topic.  You wouldn't want to have somebody who was a doctor in your focus group for a new brand of herbal cough drops.  If I was doing a focus for a political ad targeting "the average joe", I certainly wouldn't want a person who posted on daily kos in my group... they'd dominate the discussion.  Guess I disqualified myself :-)

              •  jury pool focus groups chosen this way (1+ / 0-)
                Recommended by:

                many mock trials are staged in hotel conference rooms under the guise of "focus group"

                Before Jesus, there was love. Without love, there would be no Jesus.

                by oobi on Mon Sep 01, 2008 at 12:03:52 PM PDT

                [ Parent ]

                •  I've been on a medial mock jury (0+ / 0-)

                  (Hey, they never asked if I work in Usability ;-).

                  It is often cheaper to shell out $10 or $20k on a focus group and decide if you should settle and for what amount then it is to gamble on a verdict.

                  In most cases, it is fairly obvious to participants why they are there based on the questions asked and the screener they answered.  Often times the CEO or some big money guy will come out from behind the mirrored glass when it is over and thank everybody.  Usually a "virgin" CEO is very, very stunned and impressed with how useful focus groups/usability studies are.  You learn so much about your product when you do one and usually have so much really, really good ideas to improve things when it is finished.

            •  I think the last time I heard John Zogby (0+ / 0-)

              on TDS he basically said that to get a decent sample for NH they pretty much had to call every household in the state.

              That's how few people actual answer the phone/poll.

              Given how easy it is these days to screen your calls, and how many people work unusual hours/multiple jobs, I'd say that they pretty much only get people who answer every single phone call, without paying attention to who it is.

              Which basically means somebody who's almost completely tech illiterate, or a luddite who refuses to get any new features like caller ID, voicemail, and who won't even buy an answering machine (or use it).

              They're getting a VERY skewed sample in a phone poll.

        •  The one I caught last night (0+ / 0-)

          was AARP, but not just seniors.  They had a mixed bag.  I think 80% were bald faced liars.  They weren't undecideds from either camp.  They knew EXACTLY who they were voting for.

          I don't know Luntz, but he came off like a putz, and that's putting it mildly!

    •  MSM is, and will continue to be, (2+ / 0-)
      Recommended by:
      grndrush, Amber6541

      completely out of touch with reality.

      It's becoming more obvious every day.

      John McCain: a noun, a verb and POW.

      by john in pa on Mon Sep 01, 2008 at 09:30:40 AM PDT

      [ Parent ]

    •  I am not surprised (0+ / 0-)

      that Palin doesn't do well with older voters after a retired neighbor lady of mine told me all the negatives she saw in her.

      Fox news: Even better than meth!

      by get the red out on Mon Sep 01, 2008 at 11:55:07 AM PDT

      [ Parent ]

    •  and older voters are important as they are a very (0+ / 0-)

      reliable group of voters.

      I often tell people how my father was discharged from the hospital after cancer surgery and drove himself to the polls to vote. And it was only a primary election in an off Presidential year.

      My late parents voted in every primary and every General election since 1950 and until their deaths a few years ago.

      Sen.Bob Casey said:John McCain calls himself a maverick, but he votes with George Bush more than 90% of the time...that's not a maverick, that's a sidekick.

      by wishingwell on Mon Sep 01, 2008 at 12:01:01 PM PDT

      [ Parent ]

    •  I watched the end of that focus group (0+ / 0-)

      and what I found interesting was that (1) this was held in Minneapolis immediately before the RNC convention, and (2) one person (male) admitted to having switched from McCain to Obama.

      And those FOX reporters were doing eveything they could to sway the people in that room.  It was fun to watch their frustration.

      Now, what would be really interesting would be to poll those same people after the RNC Convention.

      I am an Edwards Democrat!
      I am also droogie6655321!

      by Scubaval on Mon Sep 01, 2008 at 05:01:42 PM PDT

      [ Parent ]

Subscribe or Donate to support Daily Kos.

Click here for the mobile view of the site