Many of us -- myself included -- have made a pretty big deal about fivethirtyeight.com, Nate Silver's election projection website.
But, as some people pointed out, he doesn't really have a record of projecting national elections... until now.
So, I've crunched some numbers and compared Nate's projections to the projections from RealClearPolitics.com and Pollster.com, two other notable election projection websites.
Note that I'm only comparing their calculated margins of victory, not their actual numbers; RCP and Pollster don't allocate undecideds, so it wouldn't be fair to compare them to a site that does.
The results might surprise you.
State | Nate(538)Error | PollsterError | RCPError | Winner |
Alabama | 0.70% | 0.80% | 1.70% | Nate |
Alaska | 11.10% | 8.90% | 10.80% | Pollster |
Arizona | 3.70% | 3.70% | 4.60% | Nate/Pollster |
Arkansas | 9.00% | 9.70% | 10.70% | Nate |
California | 3.40% | 5.00% | 0.40% | RCP |
Colorado | 0.10% | 0.90% | 1.20% | Nate |
Connecticut | 1.80% | 0.50% | 2.20% | Pollster |
Delaware | 0.40% | 1.70% | 2.70% | Nate |
Florida | 0.70% | 0.70% | 0.60% | RCP |
Georgia | 0.00% | 2.60% | 1.50% | Nate |
Hawaii | 17.70% | 14.50% | | Pollster |
Idaho | 4.80% | 2.40% | | Pollster |
Iowa | 2.40% | 3.60% | 6.00% | Nate |
Illinois | 2.90% | 2.40% | 0.00% | RCP |
Indiana | 2.50% | 2.10% | 2.30% | Pollster |
Kansas | 1.80% | 0.70% | | Pollster |
Kentucky | 2.80% | 3.40% | 2.90% | Nate |
Louisiana | 8.80% | 8.10% | | Pollster |
Maine | 3.00% | 1.80% | 1.90% | Pollster |
Maryland | 0.20% | 1.40% | | Nate |
Massachusetts | 5.90% | 5.80% | 4.50% | RCP |
Michigan | 3.80% | 0.60% | 2.90% | Pollster |
Minnesota | 0.10% | 2.20% | 0.40% | Nate |
Mississippi | 2.40% | 3.20% | 2.30% | RCP |
Missouri | 0.10% | 1.40% | 0.40% | Nate |
Montana | 0.30% | 0.20% | 1.40% | Pollster |
Nebraska | 1.70% | 3.20% | | Nate |
Nevada | 7.60% | 5.30% | 5.90% | Pollster |
New Hampshire | 0.30% | 1.80% | 0.50% | Nate |
New Jersey | 0.00% | 0.90% | 0.90% | Nate |
New Mexico | 5.30% | 6.10% | 7.70% | Nate |
New York | 0.00% | 0.00% | 4.30% | Nate/Pollster |
North Carolina | 0.70% | 0.10% | 0.70% | Pollster |
North Dakota | 6.00% | 8.00% | | Nate |
Ohio | 0.50% | 0.80% | 1.40% | Nate |
Oklahoma | 5.60% | 6.80% | | Nate |
Oregon | 1.60% | 0.90% | 0.80% | RCP |
Pennsylvania | 2.20% | 3.10% | 3.00% | Nate |
Rhode Island | 4.20% | 9.40% | | Nate |
South Carolina | 0.90% | 0.30% | 1.20% | Pollster |
South Dakota | 0.30% | 0.40% | 0.10% | RCP |
Tennessee | 2.20% | 2.80% | 1.10% | RCP |
Texas | 0.90% | 1.40% | 1.30% | Nate |
Utah | 2.90% | 4.40% | | Nate |
Vermont | 10.60% | 9.00% | | Pollster |
Virginia | 1.20% | 1.10% | 0.10% | RCP |
Washington | 2.90% | 4.10% | 4.30% | Nate |
Washington, DC | 26.90% | 18.20% | | Pollster |
West Virginia | 3.30% | 1.70% | 4.10% | Pollster |
Wisconsin | 2.40% | 1.90% | 2.90% | Pollster |
Wyoming | 7.20% | 1.70% | | Pollster |
Average | 3.68% | 3.56% | 2.67% | |
Median | 2.40% | 2.20% | 1.80% | |
Best | 0.00% | 0.00% | 0.00% | |
Worst | 26.90% | 18.20% | 10.80% | |
Note RealClearPolitics' apparent good score is due to them not making projections on states where there wasn't a whole lot of polling done. These states tended to be blowout states, and blowout states are where Nate and Pollster are most widely off the mark. On the other hand, this possibly hurt RCP seriously in the rankings of who did best on each individual state.
Here are the results when those states are removed:
State | Nate (538) | Pollster | RCP |
Average | 2.35% | 2.46% | 2.67% |
Median | 2.00% | 1.80% | 1.80% |
Best | 0.00% | 0.00% | 0.00% |
Worst | 11.10% | 9.70% | 10.80% |
It begins to look better for Nate in this situation; he had the best average, and his other scores were very close to the others. However, all of them did pretty good.
Also, Nate was closest to the mark on 24 of the 51 races. Pollster was second with 20, and RealClearPolitics was third with 9. (This adds up to 53 because Nate and Pollster shared the glory on two of the polls.) It turns out that when Nate was right, he was often very right, and when he was very wrong, he was way off the mark.
Finally, Nate was the only one who had the, uhh, balls to call the tossup states for one candidate or the other before the election. In fact, he accurately predicted the winner in 50 of the 51 races. (He missed Indiana.) Pollster and especially RealClearPolitics didn't speculate on any of the truly close states.
In conclusion, I think Nate Silver did a pretty good job. He was closest on the most number of states, and he accurately called 50 of the 51 races. Not bad for his first year doing this!