Of all the analysts in all the world, 538 is the fairest of them all. Or so I thought in previous cycles.
OK, they got Michigan wrong, but hell, everyone got Michigan wrong, that was an outlier right? Well, maybe.
The table below is a list of states organized by region where 538 made forecasts before the primary/caucus. There are a number of states for which they did not make forecasts. I’ve listed their polls-only forecast on the eve of the election and their polls+ forecast which takes into account other factors. I’ve also noted whether the primary was Open, Closed or Modified, or a Caucus. A modified primary means you can either register or change your party affiliation the day of the primary.
In the last two columns, I calculate how much the actual result deviated from 538s two forecasts. I figure 3% is a reasonable margin of error.
-
Bold means Bernie outperformed by 3% (positive values)
-
Underline means Bernie underperformed by 3% (negative values)
A 3% deviation is quite substantial. If Bernie outperforms by 3%, that means the spread is 6% wider than 538 forecast.
| State |
Region |
Type |
538 polls |
538 polls+ |
Actual |
Delta Polls |
Delta Polls+ |
| MA |
North East |
Mod |
44.60% |
44.80% |
48.69% |
4.09% |
3.89% |
| VT |
North East |
Open |
87.40% |
86.70% |
86.10% |
-1.30% |
-0.60% |
| NH |
North East |
Mod |
55.60% |
57.20% |
60.15% |
4.55% |
2.95% |
| NC |
Atlantic |
Mod |
37.40% |
36.20% |
40.76% |
3.36% |
4.56% |
| VA |
Atlantic |
Open |
36.70% |
34.60% |
35.19% |
-1.51% |
0.59% |
| OK |
Appalachia |
Closed |
47.50% |
47.30% |
51.88% |
4.38% |
4.58% |
| TN |
Appalachia |
Open |
36.10% |
33.90% |
32.43% |
-3.67% |
-1.47% |
| SC |
South |
Open |
31.30% |
28.70% |
26.02% |
-5.28% |
-2.68% |
| TX |
South |
Closed |
33.70% |
32.40% |
33.16% |
-0.54% |
0.76% |
| AR |
South |
Open |
36.00% |
32.20% |
29.74% |
-6.26% |
-2.46% |
| AL |
South |
Open |
25.70% |
23.20% |
19.19% |
-6.51% |
-4.01% |
| LA |
South |
Closed |
20.20% |
17.90% |
23.12% |
2.92% |
5.22% |
| GA |
South |
Open |
30.50% |
28.90% |
28.20% |
-2.30% |
-0.70% |
| FL |
South |
Closed |
33.80% |
32.10% |
33.28% |
-0.52% |
1.18% |
| AZ |
West |
Closed |
22.70% |
|
41.14% |
18.44% |
|
| NV |
West |
Caucus |
47.20% |
46.00% |
47.29% |
0.09% |
1.29% |
| UT |
West |
Caucus |
51.10% |
|
77.19% |
26.09% |
|
| WI |
Mid-West |
Open |
50.10% |
50.20% |
56.49% |
6.39% |
6.29% |
| IA |
Mid-West |
Caucus |
44.70% |
48.30% |
49.59% |
4.89% |
1.29% |
| IL |
Mid-West |
Open |
44.30% |
44.10% |
48.72% |
4.42% |
4.62% |
| OH |
Mid-West |
Mod |
43.30% |
43.10% |
42.72% |
-0.58% |
-0.38% |
| MO |
Mid-West |
Open |
48.10% |
48.00% |
49.37% |
1.27% |
1.37% |
| MI |
Mid-West |
Open |
38.30% |
37.50% |
49.75% |
11.45% |
12.25% |
Two things stick out:
- There are a lot of bold cells (11 of 23 states), and fewer underlined cells (4 of 23 states).
- All the underlined cells are in the South (Appalachia is a wash)
Hillary outperforms 538 poll averages and forecasts in the South. Bernie outperforms everywhere else (or is within MoE).
In all, 15 out of 23 states fall outside the MoE for one or both forecasts. That’s 65%. Also note the polls plus model does seem to add some value, they are less frequently outside of our 3% MoE.
So what does this mean? Here are some reasonable guesses:
-
Bernie gains support as we approach the primary/caucus, since polls are always a few days old they under-represent his support
-
The likely voter models are wrong and Bernie is bringing in more primary voters who pollsters aren’t reaching.
- Name recognition influences poll-respondents more than it does voters.
- There are fewer primary polls, far less data, small changes in turnout can swing the result.
I think it’s the last.
Which means we should all get back to phonebanking and canvassing for Bernie.
