We have more than a year’s worth of special elections results in the Trump era to peruse now, and we’ve learned some valuable things. The most striking, of course, is that special elections as a whole really do mean something, something very important: They can tell us a lot about the outcomes of the upcoming November elections. As of this writing, the Special Elections Index is still more favorable to Democrats than it has ever been stretching back to the 1980 cycle.
The map above shows us the special election shift we’ve been seeing over the past year or so. Democrats in special elections have been overperforming Hillary Clinton’s margins across large parts of the country (blue)—but not in all parts of the country. Democrats are exceeding Clinton’s margins the most in the center of the country, where the Midwest meets the South. Meanwhile, southern New England has seen mainly Democratic underperformances.
It turns out there’s a correlation between the special election shift from Clinton’s margin in each district and Clinton’s margin in the state the district is in, as you can see in the graph below (hardcore statistics nerds can click here for regression details). Why? One plausible explanation is that deep red states have generally had one-party Republican rule for a few years now and are not terribly happy about the results. On the other hand, mixed control is generally the case in blue states like New York and Massachusetts, preventing exposure of the populace to the worst excesses of conservatism.
In the most conservative states, on the far left side, Democrats have been outperforming Clinton’s margin by around 20 to 30 points, on average. But in the most liberal states, to the right, Democrats have performed just about even with Clinton. For states that were evenly split in 2016, Democrats have performed about 10 points better than Clinton.
This is one of several correlations we see. Another one we’ve been keeping our eye on for some time is the correlation with the shift in presidential margin from 2012 to 2016. Many of the best Democratic overperformances are in districts with large decreases from Barack Obama’s 2012 margins to Clinton’s 2016 margins. Are special elections simply shifting back to 2012’s presidential margins?
If they were, all the data points would fall on the solid diagonal line in the graph above. Clearly, they’re not—there’s plenty of scatter. There’s a lot going on in this graph though; here’s a guide:
In most cases, Democrats in special elections are above the solid diagonal line, meaning they’re outperforming Obama. Indeed, in most cases they are also above the horizontal axis, with margins better than both Clinton and Obama (the orange or purple zone). There are even many cases (the orange zone) where Clinton did better than Obama, and then the Democrat in the special election did even better than Clinton. But about four in ten special elections fall into the purple zone, where results shifted towards Trump in the presidential election, but then shifted back even more in the special so that the Democrat outperformed Obama.
Just the two variables highlighted above, together, can explain slightly more than half of the variance in special election swing: Clinton’s margin in the district’s state, and the presidential shift in the district. (Adding in additional variables, of those tested, helps very little.) One way to see this in the data is by plotting just the results from the two most extreme states, Oklahoma and Massachusetts, on the same sort of graph as above.
We see for a given presidential margin change from 2012 to 2016, on average, Oklahoma has a much greater shift towards Democrats in the special election.
Although this analysis is helpful in understanding what’s going in with special elections, it’s not terribly useful in predicting the special elections margin shift. Compare the predicted and actual margin shifts:
You can see there’s plenty of room for additional factors to affect the margin shift, such as candidate quality or locally important demographics (for instance, Cuban voters in Florida, who often identify as Hispanic but, unlike most Latinos, lean Republican). Indeed, this correlation would not serve very well to predict any one individual election. But it does have its uses, which we’ll see in the next post.
Notes:
All special elections results and presidential margins can be found here. Elections where the top two candidates totaled <90% of the vote, where the top two candidates were from the same party, where one of the top two candidates was from a third party, where one of the major party candidates was recently affiliated with the opposite party, or where a close family member was running to replace a deceased office holder were not included.
Other variables that are weakly correlated with the special election shift include education levels (defined as the proportion of a district’s population older than 25 with a college degree), the proportion of white residents, the 2016 presidential margin, the 2012 presidential margin, and the share of seats held by Democrats in the state’s lower chamber (see here for more details). Many of these variables, however, are also correlated with each other to some extent; for example, education and presidential margin shift.
