A small portion of poll respondents don't seem to know where they live. At least, in DailyKos/SEIU/PPP polling, approximately 5-9% misidentify their region (after weighting by age) depending on the region. This has been a minor mystery for a while now, and, indeed, the wording of the question was changed mid-2011 to reduce possible misunderstandings, but to little if any effect; see David's post here for more. So how do we solve this mystery? [Update] Or, to succinctly and properly reframe the question as Garrett did in the comments: Everyone knows where they live. Why does the poll misreport it?
To define the problem more precisely, what is seen is that those with area codes in unambiguous regions do not always report that they live in said region. For example, of those who live in the Northeast, about 3% said they live in the Midwest, 2% said they live in the South, and 1% said the live in the West.
So why would people say they live somewhere they don't? Join me below to find out.
Could it be just a slip of the finger?
Not entirely. This year, PPP has been releasing its discarded data as well as raw data, which includes people who press a number other than 1 or 2 for the question of gender. With a few assumptions, I calculate a 0.2% maximum error rate from people unintentionally pressing the wrong button. (This may still include some people who intentionally press a number other than 1 or 2.)
More complicated questions may produce more error. For instance, when asked to input their zip codes, 0.5% hit the wrong first number if their zip code begins with 1, 2, or 3; 0.6% for 4, 5, or 6; and 1.0% for 7, 8, or 9. In this case it appears that moving to the third row of buttons is more likely to produce error, something we don't have to worry about for other questions.
Moving along to even more complicated questions, there may be issues of misunderstanding the question or not hearing it correctly due to distraction or hearing impairment. But this doesn't seem to occur very often either. Among Liberal Democrats who approve of Obama and have a favorable view of the Democratic Party - people you would expect to uniformly say they are voting for Obama - 1.1% do not say they will vote for Obama. This would set a maximum for misunderstanding a question. Keep in mind this number would also include those who accidentally press the wrong button as well.
We probably do see this sort of 'misunderstanding' error in the geographic data - among those in the West, there is a higher tendency to mistakenly identify as being in the Midwest than among those in the Northeast or South. This is consistent with people hearing 'West' instead of 'Midwest' for option 2. It can only explain a small portion of mistakes though.
Another geography-specific wrinkle is that the person reached at a given landline may be a guest or a part-time resident. However, PPP also asks the respondent to enter their zipcode. Only 0.85% of properly entered zipcodes do not match the area code, so this, too, can only explain a small fraction of the geography errors.
What's left?
Unless another reason surfaces, we are sadly forced to conclude that sometimes people do not respond truthfully to polling questions on purpose.
I'm sure this will shock readers to their very core.
So this minor mystery is solved; so what? Well, the next post in this series will address the 'what'.
___________
Beyond the Margin of Error is a series exploring problems in polling other than random error, which is the only type of error the margin of error deals with.
Previously:
Why State Polls Look More Favorable For Obama than National Polls. In the spring and summer, lack of support in Blue States was bringing down Obama's performance in national polls, while Swing States and Red States were polling about the same as 2008.
Presidential Polls Are Almost Always Right, Even When They're Wrong. How the presidential polls in red and blue states are off, sometimes way off, and how to predict how far off they'll be.
When Polls Fail, or Why Elizabeth Warren Will Dash GOP Hopes. Why polls for close races for Governor and Senate are sometimes way off, and how to predict how far off they will be.