Historically, special elections for state legislatures have tended to fly under the radar except among the more dedicated elections geeks, such as reside here at Daily Kos Elections. But as is our wont, we’ve been paying close attention to them, and we’ve seen a clear trend of Democratic overperformance this cycle. We’ve even developed a Special Elections Index to quantify that overperformance.
But there’s always the nagging question: What do special elections really mean, if anything? After all, they’re typically low-turnout affairs, in many cases for tiny districts, and limited to a subset of states (many states fill vacancies by appointment). Could they possibly be broadly representative of anything?
Now we know the surprising answer: Yes, absolutely.
The graph above shows the Special Elections Index along with the national popular vote for the House. They are similar in pattern, albeit with some notable exceptions for years with fewer data points (1998, for instance). Indeed, we can make several decent correlations between the two, and those correlations point to very good things for Democrats heading into the 2018 midterms.
What's the relationship between the Special Elections Index and the popular House vote?
It’s not unusual for politicos to spin the results of any one special election into earth-shattering significance, but careful analysts tend to be, well, more careful. Some researchers have concluded that congressional special elections are nationalized to a certain degree, especially recently; others say it is inappropriate to draw national conclusions from such elections. Some weak relationships between general election outcomes and various measures of congressional special election success have in fact been found. But all these studies have been hampered by the same problem: a lack of data. There’s simply not very many open seats in Congress each year.
We solve that problem by using not just congressional special elections but also state legislative special elections in our Special Elections Index. The Index itself is calculated by comparing special elections results to other election results for the same office in the same district (for details, see here).
The results? We see a strong relationship between the biennial Special Elections Index (which combines all the data for each entire two-year election cycle) and the popular House vote! In the graph below, years with more data are shown with larger circles.
All trend lines shown are weighted by the number of election pairs used to calculate the Special Election Index in each year. (Note that 1998 is a special case we’re saving for another post and is not included in the trend line.)
Here’s what it looks like plotting both the biennial Special Elections Index and the House popular vote over time:
Now we have our green circles following the purple House vote line pretty closely—usually a few points lower, but capturing the relative pattern. So by the time all the special elections for this election cycle are complete, probably around summer 2018, we should have a good idea what November will bring.
But what we really are itching to find out is, what does the Democratic overperformance in 2017’s special elections mean for 2018? Let’s look at the relationship between the Special Elections Index for the year prior to the election year only and the House popular vote the following year:
It’s not as tight as the relationship with the biennial index—after all, there’s less data—but it’s still pretty good! This means the mood of the electorate is, in most cases, largely set early on, and the results of special elections the year before election year do indeed have valuable information for us. And the 2017 Special Election Index value would point to somewhere between a good year and an outrageously great year for Democrats! However, there’s plenty of uncertainty; it’s best to take this as a qualitative prediction for now. (More on this next time.)
The generic congressional ballot as a standard
So you’ve seen a couple of trend lines using the Special Election Index; how do you know how much importance to give this measure? Well, here’s a comparison with a well-known indicator: the generic congressional ballot polling, averaged over the final month before Election Day:
The relationship for the polling a month prior to the election is about as good as the Special Election Index for the year before the election year! (Note that the purple circles are not sized to the same scale as the green Special Election Index circles, but the same principle applies: bigger circles mean more data.) We can also look at the trend line using an entire year’s worth of generic congressional ballot data, as shown here. It’s about the same. What about the generic congressional ballot polls in the year before the election? Well, it turns out, they’re not particularly useful. But the biennial Special Elections Index is better than any of these methods of averaging the polling data.
What is interesting, however, is that Democrats have tended to do worse in the congressional elections than in the polling (except, notably, in 2008, 2010, and 2012, which really set us up for disappointment in 2014). On the other hand, Democrats tend to do better in congressional elections than in the Special Elections Index, which is designed to have a comparable scale. One explanation is simply that Democrats are typically less apt to show up for special elections. If that’s the case, then in 2018 we might expect the data to fall below the Special Elections Index trend line, because so far we see no evidence that higher turnout in special elections results in better performance for the Democrat (and, in fact, the reverse might be true—something we’re keeping an eye on).
So if you want to get a feel for the electorate in the year prior to the election, pay attention to who people are actually voting for, and not who they say they’ll vote for next year.
Bottom line: The Special Elections Index this cycle indicates a very good political environment for Democrats right now, one that, if the last 30 years are any indication, is more likely than not to persist through 2018. With a human version of the Infinite Improbability Drive installed in the White House, however, uncertainty is higher than usual this cycle. Next time: a closer look at that uncertainty.
Data
For a detailed description of the Special Elections Index and the elections included, see this post. All elections data are available here. Generic congressional ballot polling data for 1993-2010 were graciously provided by Charles Franklin. More recent data were compiled from Huffpost Pollster and Real Clear Politics.
Update: Part 3 is here.
