Now that the political dust is settled from the 2018 mid-terms, I thought it would be a good time to see how electoral predictions stacked up against actual results. My point in doing this is that in comparing predictions to actual results one can determine the accuracy of the predictors and how much weight we want to place on their predictions in the next election cycle.
There are a number of political predictors out there (Cook Political Report, etc.), and all were amazingly accurate this election cycle as pointed out in this previous David Jarmen Post. But in this diary I have chosen to focus on how the 538 Prediction Model engineered by our own Nate Silver (used to be known as “Poblano” around these parts) stacked up against the actual results of the State Governors’, US Senate and US House races. Before I delve into the details, let me sum up Nate’s 2018 prediction accuracy in one word FANTASTIC!
So lets get right down to it starting with the Governors’ races.
2018 GOVERNOR RACES:
Predicted results from 538's Governors' Forecast the day before the election and Actual Results from CNN Governors' Results. Bolded States = Correct Prediction
State 538 Prediction Actual Result
Nevada D +0.2 D +4.1
Iowa D +1.0 R +2.7
Kansas R +1.3 D +4.5
Ohio D +1.5 R +4.2
Wisconsin D +1.7 D+ 1.2
Georgia R +2.2 R +1.4
South Dakota R +2.5 R +3.4
Alaska R +4.0 R +7.3
Florida D +4.2 R +0.4
Connecticut D +5.1 D +3.2
Oregon D +6.5 D +6.4
Oklahoma R +7.2 R +12.1
New Hampshire R +8.3 R +7.0
Illinois D + 9.2 D +15.4
New Mexico D +9.4 D +14.4
Michigan D +9.7 D +9.5
Minnesota D +10.2 D +11.5
Vermont R +11.4 R +15.0
Maine D +12.3 D +7.6
Colorado D +12.4 D +10.6
South Carolina R +13.4 R +8.0
Tennessee R +13.6 R +21.1
Arizona R +13.7 R +14.2
Pennsylvania D +15.4 D +16.8
Alabama R +16.1 R +19.2
Idaho R +16.4 R +21.6
Texas R +16.9 R +13.3
California D +17.4 D +23.2
Maryland R +17.5 R +12.7
New York D +19.1 D +22.2
Rhode Island D +20.4 D +15.5
Nebraska R +21.6 R +18.4
Arkansas R +27.3 R +33.5
Wyoming R +29.9 R +39.8
Hawaii D +30.1 D +29.0
Massachusetts R +34.0 R +32.6
Summary of Gubernatorial Races — First let’s look at the numbers. Out of the 36 races above, 538’s Model only got 4 wrong, for an accuracy rate of about 89%. Even if we don’t count the solid and likely races, those predicted with +10% leads, 538 still got 12 out of 16 correct for an accuracy rate of 75%. Truly remarkable when you consider how many of these races were toss-ups with predicted leads within the MoE. Of the races with the smallest predicted margins, 538 correctly predicted Nevada, Wisconsin and Georgia (2 Ds & 1 R), while getting Iowa, Kansas, Ohio and Florida (3 Ds & 1 R) wrong. So there doesn’t seem to be any discernible R or D bias in the tough correct/wrong predictions.
2018 SENATE RACES:
Predicted results from 538's Senate Forecast the day before the election and Actual Results from CNN Senate Results. Bolded States = Correct Prediction
State 538 Prediction Actual Result
Nevada D +1.1 D +5.0
Ohio D +11.6 D +6.4
Wisconsin D +12.8 D+ 10.8
North Dakota R +4.6 R +11.0
Florida D +3.2 R +0.2
Connecticut D +21.4 D +20.2
New Jersey D +11.5 D +10.6
Delaware D +27.0 D +22.2
Virginia D +15.8 D +16.0
West Virginia D +7.5 D +3.3
Indiana D +3.7 R +5.9
New Mexico D +18.2 D +23.5
Michigan D +11.6 D +6.4
Minnesota (1) D +21.5 D +24.1
Minnesota (2) D +9.6 D +10.6
Mississippi (1) R +15.7 R +13.0
Mississippi (2)* R +? R +7.8 (runoff result)
Missouri D +1.1 R +6.0
Vermont D +41.2 D +39.9
Washington D +22.4 D +16.8
Montana D + 4.8 D +3.5
Maine D(I) +20.0 D(I) +19.0
Tennessee R +5.3 R +10.8
Arizona D +1.7 D +2.4
Pennsylvania D +11.6 D +12.8
Texas R +4.9 R +2.6
Utah R +29.4 R +31.7
California D +15.2 D +8.8
Maryland D +35.6 D +33.9
New York D +28.2 D +33.0
Rhode Island D +28.6 D +23.2
Nebraska R +14.8 R +19.6
Wyoming R +34.0 R +37.0
Hawaii D +53.4 D +42.4
Massachusetts D +26.1 D +24.8
* Since Mississippi was a 4-way race on November 6th, 538’s prediction did not provide a margin between only the top 2 candidates.
Summary of Senate Races — Looking at the numbers, out of the 35 races above, 538’s Model only got 3 wrong, for an accuracy rate of about 91%. Also, the number of close races (i.e., those with a predicted margin under 10%) which were called correctly (Nevada, North Dakota, West Virginia, Minnesota, Montana, Tennessee, Arizona & Texas) out number those called incorrectly by about 2.7 to 1. If we look at the 3 wrong predictions (Indiana, Missouri & Florida), I would argue that Florida doesn’t even count because the Scott v. Nelson race was tighter than a tick in the end making it virtually unpredictable.
2018 HOUSE RACES:
Predicted results from 538's House Forecast the day before the election and Actual Results from CNN House Results. Bolded States = Correct Prediction.
Before I get into listing predictions and results of House races as done above for Senate and Governor races, let me make clear I am NOT going to list every House District. For the 193 Districts listed by 538 as “Solid D” (+95% D) and the 135 Districts listed by 538 as “Solid R” (+95% R), I am going to assume that the actual results from all these 328 Districts went down as predicted by 538. I have reviewed the 17 Districts predicted by 538 as “Likely D” and found that the actual results from these 17 Districts were all in favor of the Democrat, as predicted by 538 for an accuracy rate of 100%. I have also reviewed the 50 Districts predicted by 538 as “Likely R”. Here I found 4 out of the 50 Districts where 538 predicted a Republican win, but the actual results show that the Democrat won the District (NY-11, VA-02, SC-01 & OK-05). That still leaves 46 of the 50 “Likely R” 538 Predictions that were correct for an accuracy rate of 92%. BTW — The fact that 4 House races that were rated as “Likely R” (75% chance of a Republican win) went to Democrats is a clear indication of an actual Blue Wave, by any definition. So, if we are counting, so far that’s 4 D-Flips that were NOT projected.
With the above understood, below are the predicted/actual results from the 10 Districts listed as “Lean D”, the 12 Districts listed as “Lean R” and the 18 Districts listed as Toss Ups by 538’s House Forecast.
State/District 538 Prediction Actual Result
ME-02 D +2.4 (Lean D) D +1.0 (D Flip)
NY-19 D +1.7 (Lean D) D +2.9 (D Flip)
NY-22 D +0.4 (Toss-Up) D +1.4 (D Flip)
NY-02 R +4.4 (Lean R) R +6.6
NJ-03 D +0.7 (Toss-Up) D +0.3 (D Flip)
PA-10 R +2.6 (Lean R) R +2.6
PA-01 R +1.0 (Toss-Up) R +2.6
VA-02 R +2.6 (Lean R) D +2.2 (D Flip)
VA-07 R +0.9 (Toss-Up) D +2.0 (D Flip)
VA-05 R +0.6 (Toss-Up) R +6.6
NC-13 R +1.7 (Lean R) R +6.0
NC-09 R +0.4 (Toss Up) R +0.4
GA-06 R +0.2 (Toss-Up) D +1.0 (D Flip)
FL-06 R +3.8 (Lean R) R +12.6
FL-15 R +1.0 (Toss Up) R +6.0
FL-25 R +4.4 (Lean R) R +21.0
FL-26 D +1.0 (Toss Up) D +1.8 (D Flip)
OH-12 R +2.3 (Lean R) R +4.5
KY-06 R +0.6 (Toss Up) R +3.2
MI-08 D +2.5 (Lean D) D +3.8 (D Flip)
MI-07 R +1.4 (Toss Up) R +7.6
IL-14 D +3.0 (Lean D) D +4.2 (D Flip)
IL-06 R +0.2 (Toss Up) D +6.0 (D Flip)
IL-12 R +3.8 (Lean R) R +6.6
IL-13 R +3.6 (Lean R) R +4.4
KS-02 D +1.9 (Lean D) R +1.7
IA-03 D +3.2 (Lean D) D +2.1 (D Flip)
TX-32 R +2.3 (Lean R) D +6.5 (D Flip)
TX-07 D +0.4 (Toss Up) D +5.0 (D Flip)
MN-01 D +0.8 (Toss Up) R +0.4
NE-02 R +1.4 (Toss Up) R +2.0
NM-02 R +0.8 (Toss Up) D +1.8 (D Flip)
UT-04 D +1.8 (Lean D) D +0.2 (D Flip)
WA-08 D +2.8 (Lean D) D +4.8 (D Flip)
WA-03 R +3.8 (Lean R) R +5.4
CA-25 D +2.4 (Lean D) D +8.6 (D Flip)
CA-39 D +1.4 (Toss Up) D +3.0 (D Flip)
CA-48 D +1.2 (Toss Up) D +7.0 (D Flip)
CA-45 D +2.2 (Lean D) D +4.0 (D Flip)
AK-AL R +2.8 (Lean R) R +6.6
Summary of House Races — Before I get into the #s, a couple of caveats. As of this writing, CNN has not yet called the CA-21 race for the Democrat, but I am considering it won since other news agencies have called it. Also, the Republican in the ME-02 race is in court challenging Maine’s instant run-off law, but I’m assuming he will lose that challenge. Other than that, I think the other races are a done deal, as far as I know.
Now getting down to the numbers, out of the 435 House races, 538’s Model got 12 wrong (4 ranked “likely”, 3 ranked “lean” and 5 ranked “toss up”), for an AMAZING accuracy rate of about 97%. Even if we simply look at the races above ranked as “leans” and “toss ups” (those with predicted margins under 5%), 538 only got 8 races out of 40 wrong for an accuracy rate of about 80%. Once again, AMAZING!
Although the above accuracy rates are amazing, what really drops ones jaw is astonishing accuracy of the 3 of D Flips. Looking at 538, Nate predicted that we would pick up anywhere from 21 seats on the low end, to 59 seats on the high end, with a predicted average pick up of 39 seats. With the caveats I mentioned above, we actually picked up 40 seats, just 1 seat above Nate’s predicted average. I sit here STUNNED IN AMAZEMENT!
So I will end by asking you all to give a BIG round of APPLAUSE for Nate Silver (Poblano) and his AMAZING 538 Prediction Model!
Hands Clapping!
