Greetings polling and statistical geeks! I thought it would be interesting to have a discussion about the methodology around testing political messages. This is something that has been kicking around the back of my head ever since I was polled a few times over the past several months.
I work with survey data in my job, so I'm familiar with issues around methodology and question wording. I've been polled recently both by Rasmussen and by my local school board. In both of these polls, I was asked some questions about whether, if I heard a certain piece of information, it would make me more likely to support a particular policy or initiative.
The problem I have with this structure of the question is that if your mind is made up already, you don't really know how to answer, and your response is going to be of dubious worth.
In the case of my local school board poll, the question was around whether I would support a local parcel tax that would go to the school district. Now, I have two kids who are currently students in our local school district, and I am a huge supporter of public schools. I don't care how big they want to make the tax, I am going to vote yes on it no matter what.
So when the pollster (actually it was automated) calls and says, "If you heard that seniors can opt out of the tax, would that make you more likely to support it?" - or another similar question - I have no way to answer this.
I am 100% going to vote for the tax, no ifs ands or buts. Therefore, my answer technically should be no; I cannot possibly be more likely to vote for it no matter what message they are testing. On the other hand, it sounds like a nice message, so maybe I should say yes.
I realize that when you are doing the analysis, if you ask an initial question about their plans on how they will vote, you can throw out the people who said they will vote for it and only look at the responses within the group who say maybe or no on how they will vote. That may be how they analyze it. But why even test the message among those who are yeses to begin with?
Is there a better way of asking this type of testing question? If not, if they are doing polling and plan to use this method, shouldn't they do a 'skip' on people who say they plan to vote for the item to begin with?
What do you think? Thanks.