The Trouble with Polling and Slant

Chris Cilliza, writing in the WaPo, writes a story based on a very flawed poll and then twists the results:

Democrats fumed last week at Vice President Cheney's suggestion that criticism of the administration's war policies was itself becoming a hindrance to the war effort. But a new poll indicates most Americans are sympathetic to Cheney's point.

Seventy percent of people surveyed said that criticism of the war by Democratic senators hurts troop morale -- with 44 percent saying morale is hurt "a lot," according to a poll taken by RT Strategies. Even self-identified Democrats agree: 55 percent believe criticism hurts morale, while 21 percent say it helps morale.

The results surely will rankle many Democrats, who argue that it is patriotic and supportive of the troops to call attention to what they believe are deep flaws in President Bush's Iraq strategy. But the survey itself cannot be dismissed as a partisan attack. The RTs in RT Strategies are Thomas Riehle, a Democrat, and Lance Tarrance, a veteran GOP pollster.

Two quick points - what did Cheney actually say:

Following President Bush's lead, Cheney praised the character of Rep. John Murtha even as he voiced strong disagreement with the Pennsylvania Democrat's proposal last week to pull out all U.S. troops.

"He's a good man, a Marine, a patriot -- and he's taking a clear stand in an entirely legitimate discussion," Cheney told the American Enterprise Institute, a conservative think tank. Cheney, who represented Wyoming in the House of Representatives in the 1980s, called Murtha "my friend and former colleague."

Hmm. Nothing there about morale of the troops. How about this?

If Cheney was muted in his words on Murtha, he held no punches in criticizing the administration's political opponents, particularly Senate Democrats who voted in support of the October 2002 resolution authorizing Bush to use force against Saddam -- and who are now among the most vocal war critics.

"What is not legitimate -- and I will again say is dishonest and reprehensible -- is the suggestion by some U.S. senators that the president of the United States or any member of his administration purposely misled the American people on prewar intelligence," Cheney said.

As to suggestions that Bush, Cheney and others in the administration manipulated intelligence to make a stronger case for invading Iraq, "this is revisionism of the most corrupt and shameless variety," Cheney said.

Well, nothing about the morale of the troops there either. So where does the "morale question" come from? Sounds completely made up to me.

But the poll question itself is simply very bad polling. I'll explain why on the flip.

The question asked:

Thinking about the war in Iraq, when Democratic Senators criticize the President's policies on the war in Iraq, do you believe it helps the morale or hurts the morale of our troops in Iraq? [a lot or some]

Now, the problem with the question is blindingly obvious. Why would it effect MORALE AT ALL? And the results bear out what a ridiculous question it was. Fully 17% said NOT SURE/REFUSED. Why such a high refusal rate? Because the respondents never even considered such a ridiculous question. The question is utterly leading and invalid. It forces a result. To wit, it is a push poll.

Another problem is the length of the question. Are respondents saying that Dem criticism hurts morale, or are they saying the President's Iraq policy is hurting morale? When criticizing the President's policy on Iraq, does it help or hurt morale? The question is very poorly phrased.

The other problem is the grouping into hurts/helps a lot/some. 44% say a lot. 26% say some. 38% say they are Conservative. 31% say they are Republicans. You get my drift. What does this poll really mean? Indeed, why didn't this poll ask what respondents think of the President's Iraq policy?

I think it is clear. This was a hit poll. Designed to hit Dem criticism of the war. Was it partisan? I dunno. But whether designed for partisan reasons or just to make a splash, it is simply not a profesional poll.

Let's consider a poll result we like, a WaPo poll that found 57% of Americans believe BushCo misled on Iraq prior to the war:

Before the war began, do you think the Bush Administration did or did not intentionally exaggerate its evidence tht Iraq had [WMD]?

Is this not susceptible to the same criticism? Partly, but this question at least addresses what Cheney is ripping Dems about. Moreover, WaPo did an alternately worded question:

In making its case for war with Iraq, did the Bush Administration (told the American public what it believed to be true) or (intentionally misled the American public)?

The result? 52% believe BushCo intentionally misled.

More importantly, the question did not come out of th blue. It was not wholly leading to ask it.

In the end, all political polls suck. Non leading open ended questions are anathema to political pollsters. Sampling is a disaster. The methodology sucks. But this RT poll is especially egregious.

If this is what we can expect from the Cook Report's monthly polls [The Cook Political Report was not involved in the preparation of this survey], better left unread.