(There's a great deal of mystery about how polling is conducted, so I asked Del Ali, president of Research 2000, to discuss the mechanics of the polling process. Hopefully, this is not just educational, but helps squash some of the more ridiculous assertions made by critics of our commissioned polling -- kos)
HOW A POLL IS CONDUCTED
By Del Ali, Research 2000
So, how is a poll conducted? We will go over step by step precisely how R2K conducts a poll:
1) Questionnaire or survey instrument: This is the list of questions that will be asked in the poll.
First of all, it is vital to ask the question as fairly and objectively as possible. The exact wording of the questions and their order in the questionnaire are obviously important, as this can be the most controversial part of any poll. To provide full transparency, we publish the exact questions we ask, in the order asked.
In addition, R2K and our clients make a point of releasing the full results of every poll conducted, instead of publishing just the positive results and hiding less favorable numbers. I am critical of candidate polls--for both Democrats and Republican--that only release the horserace results, but refuse to release either the rest of the results, or even the order in which the questions were asked.
What must be emphasized is that it is important to be objective and have the facts straight when asking any issue question. Adjectives cannot be used in asking such questions, period. A question about the death penalty, for example, needs to be asked fairly and straightforwardly, with no embellishment: "Do you favor or oppose the death penalty?"
2) Finding the respondents: After our survey instrument is ready to commence, we then CATI the survey. This means simply that we put this into our Computer-Assisted-Telephone-Interview system.
Those interviewed are selected by the random variation of the last four digits of telephone numbers, whether this is a national poll, statewide, congressional district, etc. A cross-section of exchanges are utilized in order to ensure an accurate reflection of the area being polled. People often note that they've never been polled. Well, the first law of polling is that every single individual in a designated population must have an equal chance of being selected as part of the polling sample. In other words, if one was conducting a poll among likely voters in the United States, the validity of the poll comes into question unless every single adult 18 years or older has an equal chance of being called randomly.
3) Asking the questions: First we screen respondents depending on the criteria the client requires. So if we need to zone in on likely voters, we screen for likely voters (more on how we do that below). Then we ask three demographic questions. The first is the gender of the respondent. The second is political party identification, which is simply, "Politically, do you consider yourself to be a Democrat, a Republican, an Independent, or of another party?" (Keep in mind that there are many states in our nation that do not have party registration.) The third question we ask is the age of the respondent.
Then we go directly to our name recognition question(s). This is a two-part question in which we ask the respondent over the telephone if they recognize a name being polled, say, "Markos Moulitsas".
If they say do not recognize Markos, we go on to the next name on the list. If they do recognize Markos, we than ask the respondent if they have a favorable or unfavorable opinion of him. If they respond they have no opinion of him or if they respond favorable or unfavorable, we either move on to the next name or we measure the intensity of that answer by asking the respondent if that is very favorable or very unfavorable.
After we go through all the names on the list, we then go to the horserace question. That is simply asked: "If the election for President were held today, would you vote for Barack Obama , the Democrat, or John McCain, the Republican?" After the horse race question, we then move on to other issues, such as asking job approval of the President, the Democrats in Congress, the Republican in Congress, or any of the national leadership figures commonly polled.
If we were conducting a statewide poll, we could do job approvals among the Governor, both U.S. Senators and the Legislature of the state. After this, we could ask several issue questions that pertain to both state and national issues.
Finally we ask additional demographic questions. Keep in mind we already asked three of them -- sex, party affiliation, and age. We now ask the rest, such as geography and race.
4) Determining likely voters: How do pollsters define "likely voter" and how are they selected?
This is a major source of controversy when it comes to the accuracy of polls. We will simply say this: we use several screeners in determining how the likely voter is selected. We already shared one of the most important screeners -- party identification. While we will not share the actual wording of our screeners because they are proprietary and they are what make us successful, we will share two examples of those screeners:
SCREENER: Which of the following best describes what you will do next Tuesday when it comes to the November election:
I will definitely vote-PROCEED
I am pretty certain I will vote-PROCEED
I am not sure if I will vote-TERMNATE
I will not vote-TERMINATE
I am not sure-TERMINATE
SCREENER: How often do you vote when there is a statewide election?
I vote always-PROCEED
I vote almost always or most of the time-PROCEED
I vote some of the time-TERMNATE
I rarely vote-TERMINATE
I don't vote-TERMINATE
5) Additional information on poll of Republicans. We need to address a few items pertaining to the recent national poll we just completed for the Daily Kos among 2,000-plus self-identified Republicans nationally.
We started with a screener that was, quite simply, "Politically, do you consider yourself to be a Democrat, a Republican, an Independent, or of another party?" In other words, if they stated they were a Democrat, an Independent, something else or told us to go to hell, they were not part of this 2,000+ sample. It looked like this:
Politically, do you consider yourself to be a Democrat, a Republican, an Independent, or of another party?
DEM 1-TERMINATE
REP 2-PROCEED
IND 3-TERMINATE
OTHER 4-TERMINATE
DK 5-TERMINATE
After these self-identified Republicans were selected, we then asked them: "In the 2010 Congressional elections will you definitely vote, vote, not likely vote, or definitely will not vote?" As was reported, 8% of this sample stated that they are not likely to vote, 5% stated they definitely will not vote and 4% were not sure if they would vote, while 83% stated that they would definitely vote or vote.
The purpose of the poll, as clearly stated, was to interview self-identified Republicans on various issues. Each of the questions asked on the poll have been released via Daily Kos, as well as the order in which the questions were asked and the demographic information collected. These were straightforward and objective "favor or oppose" or "yes/no" questions, such as "Do you favor or oppose the death penalty?" and "Do you think Barack Obama is a socialist?"
Margin of Error
The most common complaint we hear from the public is: “Nobody has ever polled me.” For millions of people, that will be true throughout their lives. However, these individuals, while they may never be interviewed by R2K or any other firm, have an equal chance of being selected as the lucky respondent who ends up qualifying to participate in the poll. The beauty of polling is that one does not have to contact and interview every single likely voter in the United States to see where the 2008 Presidential election stands a week before the election. In fact, we only have to interview 400 likely voters to get an accurate refection of where the race stands. However, if we want to learn where the race stands among the various subgroups within the poll (for example, 18-29 year olds or Latinos), a larger overall sample is recommended because the sub-group of the 18-29 year old and Latino voter in a 400-voter sample would be very low. So, the higher the overall sample, and the higher the subgroup overall numbers, the lower the margin of error. The MOE is as follows for each sample size:
SAMPLE SIZE MOE
400 5%
600 4%
800 3.5%
Using the MOE of 5% in our 400 sample size, this means if Barack Obama yielded 50% as his level of support in the poll, his actual level of support will not be higher than 55% or lower than 45% if every single likely voter in the nation had actually been interviewed. That 95% confidence level simply means that we are 95% certain that this principle is true.
