Public Policy Polling, a Raleigh, NC-based pollster with Democratic sympathies, has a well-earned reputation as one of the best pollsters in the business. In poll after poll, they have put out accurate results, while making their demographic crosstabs freely available for all to see (unlike some other pollsters...like a certain pollster that starts with "R" and ends with "asmussen"). However, like every good pollster, there are some elections which they don't do so well on, such as the 2010 special election for Pennsylvania's 12th congressional district, or the 2010 Nevada senate race. Nevertheless, they deserve accolades for putting themselves out there.
In the days before last night's elections, PPP put out polls for eleven races. In this diary, I will compare PPP's results with the actual outcomes of these races. Numbers past the flip.
It should be noted that polling is hard, but polling off-year elections is especially hard. Nobody knows for sure who will turn out, which makes it hard for pollsters to get a representative sample. Also, people who are undecided in polls aren't necessarily going to split 50-50. If a race goes 54-46 Dem and the pollster had 46 Rep, 43 Dem, and the rest undecided, but most of the undecideds were Democrats, then that shouldn't really be counted against the pollster. So there is somewhat of a subjective element to this.
Note that in Mississippi polls are "only" 98% reporting, but I doubt that last 2% will change much.
For the sake of consistency, when I type out numbers, the good guys will always be first. (MS Initiative 31 regarding eminent domain is sketchy regarding which side is which, but I decided to make Yes the good guy for simplicity's sake, since outgoing R gov Haley Barbour was for No)
MS Initiative 26 (Personhood)
PPP: 44-45 Yes (Yes +1)
Actual: 58-42 No (No + 16)
Error: Yes +17
MS Initiative 27 (Voter ID)
PPP: 29-64 Yes (Yes +35)
Actual: 38-62 Yes (Yes +24)
Error: Yes +11
MS Initiative 31 (Eminent Domain) - wrongly identified as 28 in PPP Poll
PPP: 51-39 Yes (Yes +12)
Actual: 73-27 Yes (Yes +46)
Error: No +34
MS Governor
PPP: 40-54 Bryant (R+14)
Actual: 39-61 Bryant (R+22)
Error: D+8
MS AG
PPP: 52-39 Hood (D+13)
Actual: 61-39 Hood (D+22)
Error: R+9
MS Treasurer
PPP: 35-54 Fitch (R+19)
Actual: 38-59 Fitch (R+21)
Error: D+2
IA-SD-18
PPP: Mathis 52-46 (D+6)
Actual: Mathis 56-43 (D+13)
Error: R+7
OH Issue 1 (judicial age)
PPP: 48-27 No (No +21)
Actual: 62-38 No (No +24)
Error: Yes +3
OH Issue 2 (collective bargaining rights)
PPP: 59-36 No (No +23)
Actual: 61-39 No (No +22)
Error: No +1
OH Issue 3 (health care)
PPP: 35-49 Yes (Yes +14)
Actual: 34-66 Yes (Yes +32)
Error: No +18
Maine Question 1
PPP: 48-44 Yes (Yes +4)
Actual: 60-40 Yes (Yes +20)
Error: No +16
Let's say you have a scale where 50 points means a race was nailed on the head. For each point they were off in the bad guys' direction, you subtract a point, and for each point off in the good guys' direction, add a point. If you assign such a value for each race and average them, you get ~44, meaning...
Average partisan error: +6 Bad Guys.
Or, you could just average the error without paying attention to which side it was for. This would give you...
Average total error: 11.45.
Conclusions:
Not one of PPP's best polling days. But the fact that they even put themselves out there for this many races says a lot about them. Plus, I would rather have my expectations lowered and then be pleasantly surprised on election day, which is what happened here thanks to PPP. Also, PPP can take pride in its polling of Ohio Issues 1 & 2 and the Mississippi Treasurer race, all of which it nailed. The fact that PPP did so well on Issues 1 and 2 while doing so poorly on Issue 3 raises the question of whether the poll question was poorly phrased or whether voter confusion over the strangely-worded ballot language meant that respondents gave one answer to the pollster and then another answer on their ballots.