As we all discovered after the voting polls closed in the 2016 general election, statewide surveys of key swing states were wrong in predicting a victory for Hillary Clinton.
This has created skepticism about polls in general yet the reasons why state polls can generate misleading results are related to two issues that are still at play in 2020 state-by-state primary polls:
- Deciding who is a "likely voter" in state polls widely varies from poll to poll.
- Some state pollsters don’t adjust their polls to reflect that college graduates are more likely to take surveys than adults with less formal education.
What You Need To Know About Polls Of Likely Voters.
Some polls survey all adults; others survey registered voters. Still others survey "likely voters." Deciding who is a "likely voter" is much trickier than it sounds. … "If pollsters disagree on who constitutes a likely voter, than they will also disagree on who’s winning," the Washington Post’s Philip Bump recently wrote.
The "likely voter" category is loosely defined, so it can inspire confusion. … The further you are from the election, the harder it is to accurately predict who’s likely to vote. "Likely voters are hard to define even before an election, but a year out it’s even harder," said Emily Swanson, Associated Press director of public opinion research.
… Another challenge specific to the 2020 election is that most experts expect a historically high level of voter turnout. This means that past voting patterns may not be as predictive of who will be a likely voter for 2020 as in the past. "Every indication we have is that 2020 turnout will be bonkers," CNN’s Agiesta said. "If you’re strictly limiting your likely voter model to someone’s past voting record, you are missing something." (Politifact, November 21st)
College Graduates Are More Likely To Participate in Polls.
Looking ahead to 2020, election junkies can expect to see some high-quality polling done at the national level and in many states. But make no mistake: High-quality state-level polling in the U.S. remains sparse and underfunded. …Somewhat alarmingly, a number of state pollsters continue to ignore the overrepresentation of college graduates … (Pew Research, November 19th)
(And)
"The industry has known for years that they have to adjust for education, because for whatever reason, people with more formal education are more likely to respond to polls than people with less," said Courtney Kennedy, the director of survey research at the Pew Research Center. …. Kennedy said that since 2016, more polls have been weighting for educational attainment, but such changes haven’t been universal. "Have all the pollsters fixed this since then? Some have; many have not," she said.
If a poll does not weight for education, then the reader should beware. You can find that in the fine print. (Politifact, November 21st)
Understanding Margin of Error (MOE) In State Polls
When a state poll has a margin of error of 3 to 4 (+/-) percentage points, a candidate that appears to be leading may not actually be in the lead.
It’s entirely possible that the candidate who appears to be trailing the leader by 3 or more points may actually be slightly ahead of the leader. These polls should be considered too close to call.
Add to this, to shrink the margin of error to 1%, polls need to survey 4,000 to 5,000 respondents. A sample size in this range is usually too costly for polls conducted by state newspapers and universities.
When looking at a state poll with a small sample (400-500) of only Likely Voters/LV (as discussed above), it’s far less accurate than larger polls of Registered Voters (RV).
What About Opt-In Online Polls With Large Samples?
When it comes to opt-in online polls, enormous sample sizes aren’t necessarily a sign of quality. Given that opt-in surveys are so cheap to field, it’s not hard to drive up a sample size to provide the illusion of precision. But Center research suggests that an 8,000 person opt-in survey is not necessarily more accurate than a 2,000 person survey.
The bottom line for now is that, at least in our own explorations, “even the most effective adjustment procedures were unable to remove most of the bias”9 from opt-in polls. (Pew Research, November 19th)
Pay Attention To How Political Writers & Talking Heads Cite Polls.
It’s clear the media loves a horse race and will devote more time discussing those candidates who appear to be leading in the very latest poll released.
However, for a clear picture of any poll, it’s worth the effort to actually check the cross-tabs of each poll, especially when writers and talking heads fail to mention the following:
- Who was polled: all U.S. adults, only registered voters, or likely voters.
- The Margin of Error (MOE).
- The total number of respondents.
- If the poll was conducted online, via landline or cell phone, and in what combination.
I hope this helps provide a better understanding of state polls in the 2020 primary.
Though some experts say nationwide polls in the 2020 primary are more accurate than statewide polls, these can also be misleading for the same two reasons stated at the beginning of this piece.
***
Sources:
https://www.pewresearch.org/methods/2019/11/19/a-field-guide-to-polling-election-2020-edition/
https://www.politifact.com/truth-o-meter/article/2019/nov/21/politifacts-guide-understanding-public-opinion-pol/
https://medium.com/pew-research-center-decoded/5-tips-for-writing-about-polls-9cb0596ff28
https://www.cnbc.com/2019/11/05/why-experts-are-getting-presidential-election-polls-wrong-again.html