This one's for the home team and the night owls who won't be able to sleep a wink through tomorrow night anyway. GO ELIZABETH.
Yesterday, the Boston Herald released a last minute poll conducted for UMass-Lowell by an independent contractor, showing the Massachusetts Senate race in favor of Scott Brown 49-48. Now this was not actually good news for our truck drivin' man, because the same poll had him up by 49-45 six weeks ago i.e. Brown had stalled as Elizabeth was zooming past him. In any case, the alarm went up around here: "Outlier! Outlier!" (this being the Herald, I'm not sure they weren't shouting "OUT AND OUT LIAR"), so we went to work.
Whenever we see an "outlier" poll (and these two run against the grain of other recent MA polls), we try to get behind the headlines to look at the underlying questionnaire and, more importantly, the demographics of the respondents. These are not always easy to get to (see Note 1 below), but in this case, they were available (link provided in the text above). We then check the demographics against two gold standards: (a) the 2008 CNN Exit Polls (b) the 2010 U.S. Census Bureau data. If there are significant discrepancies between the poll demographics and the standards, we use various methods to adjust and "normalize" the results for a more accurate projection.
The Boston Herald Poll
The three categories in this poll which varied the most from the corresponding categories for Massachusetts in the CNN 2008 Presidential Election Exit Poll and the 2010 US Census are shown below, with the biggest offenders highlighted.
(Data in cells are percentages. Row total in category may not equal 100 because of rounding)
Source: Boston Herald Poll, CNN 2008 MA Senate Exit Poll, US Census Massachusetts 2010
Analysis
The "thumb on the scale" here is not egregious but one still can see the Republican thumbprint: bias your sample population against the young, increase the number of seniors, load it up with white folk and rich folk, and voilà - Brown leads Warren.
Well, all that can be adjusted fairly easily, by unbiasing the sample to resemble more normal voting demographics -- we used the 2008 CNN numbers for our model -- and recalculating. Since the pollster did not see fit to provide cross-tabs, we derived them by cross-correlating (don't ask!) the two Herald polls and getting best fits recursively (we said, don't ask!!). The results are hardly surprising.
Boston Herald Poll: Warren 48.00 Brown 49.00
Remove Age Bias: Warren +3.00 Brown +0.00
Remove Race Bias: Warren +4.00 Brown -2.00
Remove Income Bias: Warren+2.00 Brown -2.00
We used the maximum of these effects to come up with the likeliest outcome at the end of this day:
Sen. Elizabeth Warren: 52% ex-Sen. Brown: 47%
GO ELIZABETH. But we've got to get out the vote today. I'll be door-knocking and providing transport, but gotta get some sleep first. What about you? GOTV.
Conclusion
The Boston Herald Poll was not terribly bad. But since UMass-Lowell is a member of the American Association for Public Opinion Research (AAPOR)Transparency Initiative, it is surprising cross-tabs were not included. Without this, the survey cannot be independently checked for reliability and must be graded C-.
Note 1: Most reputable polling organizations will provide the backup questionnaires, demographics and cross-tabs for published polls. Most University affiliates (UMass-Lowell, Suffolk, Marist, Franklin & Marshall etc.) are good in this regard, the commercial pollsters less so. Most are signatories to the AAPOR Transparency Initiative, but significantly
NOT YouGov, Susquehanna lowlighted in my previous diaries) or Rasmussen and its parent Pulse Opinion), notorious for its Republican tilt. Be warned.