I have been seeing a lot of hand wringing about why the 538 model is so “pessimistic” on Clinton. Comments complain about how a model predicting a 33% chance of a Trump victory is somehow a bad thing.
So as someone who uses modeling and statistics like this professionally, let me give you my take on the 538 model. Based on what Nate and his team have discussed in their podcasts, I would and have trusted my life to the exact methods they are using. In fact, so has everyone who has ever flown in an airplane. Here are my thoughts:
The election is a binary event for each candidate. Either Tump will be sworn in as POTUS or he will not. That means that as far as reality is concerned, he has either a 100% chance of becoming president or a 0% chance. The models are irrelevant in that regard. The percentages output by any honest modeler are either a solved for parameter in a statistics equation or a proportion of monte carlo simulations where one outcome appears.
1) The 538 model errs on the side of uncertainty.
There are only so many presidential races in the “current era”. In fact, if you were to flip a coin for all of the presidential races since 1992, you would have a better than 1% chance your coin would correctly predict them all. Not only that, but the country is constantly changing. The Pennsylvania of 2000 was not perfectly predicted by the Pennsylvania of 1996, so each “experiment” cannot be considered “the same” statistically.
The model uses a technique called the “T” distribution to compute results; which is a set of statistical techniques used to model systems with sample sets as few as 3 samples. The other side of the coin is that when making specific predictions, such as “Hillary Clinton will earn X or more votes to Y% confidence”, the T distribution will always yield a wider margin and fewer “at least” votes for Hillary.
As an aside, the big margin is also useful. If you’re trying to catch a corona satellite film canister and need to know how large of a naval cordon to set up around the target point, an intentional error to the large might be wise to keep your intel out of the hands of the Soviets. It becomes a balancing act between the cost of the test launches to collect samples and the cost of the naval operations to run the cordon.
2) The model is based on a particular set of inputs.
The 538 team has commented that they are not using other data indicators such as primary participation, yard sign counts, ad responses, and others. Sampling theory is well published, well documented and has proven predictive value.
The 538 team has commented that the number and quality of polls are lower this year than last cycle, and that a trend in fewer public polls is continuing. Fewer samples across lots of swing states makes for a very sparse data set. This compounds the lower and lower response rates to polls.
The 538 team also notes that they apply a “house effect” correction to polling firms based on past performance relative to the actual election result. The wisdom of such a correction is debatable, since polling firms update their methods between cycles to correct their own house effects. However, it does fully explain how a poll showing Clinton +1 could reduce the chance of a Clinton win in the state.
Another phenomenon, not noted by the 538 team but here on Dailykos is that many polls seem to be strategically released to manipulate poll aggregates. This has gone on for quite some time, but with the dwindling number of polls, we may have crossed a threshold where two or three outfits releasing several intentionally skewed polls could throw a sensitive model off. In fact, it is possible for a pollster to have tested the 538 model and released results in an attack on it.
3) The dynamics of this race are unusual.
Clinton has a powerful, across the country coalition. Polling shows her cutting Romney’s margin in HALF in Texas, while maintaining a national lead on par with the 2012 Obama margin. Those extra voters have to come from somewhere and that reduced her margins proportionally in Blue states.
Combine this with low quality data and poof! the T distribution method shows “high” (10% or better) chances of Trump victories in states he should not have a chance in. Throw in lots of bernoulli trials via monte carlo simulations and the Trump odds are amplified greatly.
Conclusion:
When you read the model outputs, what you really need to be taking away is that The model is flagging itself as having poor predictive value. The model has a reasonable chance of being “correct”; in this case “correct” meaning that 40 states have their vote margins fall within the “margin of error” bounds the model predicts. Note that those 80% bounds are very wide, and are consistent with the same Clinton victory that the other predictors are describing. That being said, the real takeaway is how wide those margins of error really are.
The model itself is just a set of outputs of algorithms with a set of inputs. In this case, measuring public polls and weighting them for track record and unbiasing them by past performance is failing to show a winner in this election. This is almost certainly due to a breakdown of the post WWII public polling paradigm.
Relax. GOTV. Call an extra person because I am hatched so I can’t.