Unlike U.S. Senate or gubernatorial races, individual U.S. House races are infrequently polled, while those that are polled typically are from a limited subset of competitive races. To get a sense of the parties' national standing, pollsters resort to testing the generic ballot instead and will often ask respondents a variant of "Would you vote for the Democratic or Republican candidate in your congressional district?" without naming specific candidates. These results inform opinions of predicted net seat change, can factor into statistical models, and are subsequently compared to the actual results to judge pollster accuracy.
But just how accurate is this polling overall compared to the actual popular vote? Let's take a look at the polling average of the spread between Republicans and Democrats from the last several election cycles via the aggregating website RealClearPolitics. Note I've eliminated minor party vote totals for consistency later and many pollsters don't give a generic option for them anyway, though the difference is minor:
Turns out the polling averages were frequently wrong by more than just a trivial amount and that error didn't randomly vary from cycle to cycle either. While the averages overestimated Democrats in 2004 and particularly 2006 and by larger margins than they overestimated Republicans in 2008, 2010, and 2012, we should expect the trend of the last three cycles to continue at least in the short term, however take caution since our sample size is quite small.
This recent Republican lean is partly because pollsters are increasingly having trouble reaching more mobile voters, those who lack landlines, and those whose preferred language is not English, all demographics which skew heavily younger and less white and consequently lean Democratic. Even pollsters who contact cell phones are not immune from large misses, as happened to Gallup in 2010 and 2012, because they underestimated the non-white share of the electorate, as pollsters typically weight demographic factors such as race to counter low response rates, but this can only do so much if the unweighted subsample itself is faulty.
So what does this mean for 2014 and why does the congressional popular vote matter in our democratic system? Let's head below the fold to find out.
In large part because the U.S. House district playing field is biased toward Republicans, with the 226 districts having voted for Romney to just 209 for Obama, and because 2014 is a midterm year with an unpopular president, conditions which have historically almost always seen the president's party lose seats, there is a near universal consensus among pundits that control of the chamber is not in play. However generic ballot polls point to wildly divergent outcomes with some such as CBS predicting a repeat of the 2010, while others like Pew have Democrats narrowly ahead.
There is one source of generic ballot polling error that has largely gone unnoticed though, even as pollsters bear increasing scrutiny over their methodologies and samples: that of uncontested districts where one of the two major parties does not even have a candidate on the general election ballot. Why is this a problem? Think of it this way: modern Republican and Democratic presidential nominees are on the ballot in every state, but what if that weren't the case and candidates sometimes didn't bother in states where they couldn't win?
Let's suppose that Romney didn't bother getting on the ballot in Maryland while Obama wasn't on the ballot in Texas. Ignoring minor party candidates and switched votes, simply removing the loser's vote total in those states causes the national popular vote to drop from nearly a 4 percent Obama win to just over 2 percent. It wouldn't make sense to judge a real pollster in 2012 as more accurate who had a final spread of 2 percent than one with 4 percent if both were drawing from a national sample, yet this is exactly what happens with the generic ballot because pollsters can't adjust for actual candidates on or off the ballot.
Therefore, when we think about generic ballot polls we should remember they are operating under the assumption that all 435 districts would have one Democrat and one Republican on the ballot. If these uncontested districts were distributed evenly between the parties and by partisanship this wouldn't be a problem, but that isn't the case either.
Note that some states such as Florida tally zero total votes if only one candidate runs. It's easy to see the wave elections of 2006-2010 when there was a large disparity between the parties, but uncontested seats can still skew the outcome even in years such as 2012. That's because despite leaving only 25 seats uncontested to just 20 for Republicans, those 25 districts were much less hostile to Democrats than the other 20 were hostile to Republicans. Repeating the previous exercise of removing the presidential loser's vote total in these 45 districts would narrow the national popular vote from just under 4 percent to 2.7 percent.
In trying to account for the skew these districts have on the national popular vote, the most logical method is to utilize the presidential election vote by district, which requires there to be sufficiently strong relationship between presidential and congressional performance nationally. Looking at all the districts where both parties competed and removing minor party or independent candidates for consistency, a linear regression finds a clear relationship between the two. The year 2012 was exceptionally strong by historical standards with a coefficient of determination, or R² value, of 0.91 (values range from 0-1 with 1 being a perfect relationship).
While there are multiple mathematical methods for estimating what the house vote would be if every district had at least a name on the ballot for both parties, I wanted to keep it relatively simple. First, I sorted all districts into three categories, those with Democratic incumbents, Republican incumbents, and open seats. Then I divided the Democratic House vote total by the Democratic presidential total, took the median value in each category, and multiplied it by the Democratic presidential vote total in each corresponding uncontested house district and then did the same for the Republican vote.
This method is imperfect and produces the wrong winner in a few uncontested districts won by opposite party presidential and house candidates, but on the whole it works reasonably well enough to give you a general idea of what the popular vote might have been had there been a name on the ballot for both parties:
The Democratic popular vote margin was overstated in their 2006 and 2008 wave elections in large part because they fielded more candidates and vice versa for Republicans in 2010. Furthermore the 2006 polling error becomes particularly acute with a miss of more than 4 percent. In 2012 Democrats had a clear popular vote win any way you look at it, while the same goes for Republicans in 2004 with a minor readjustment in favor of Democrats, however due to only having access to presidential vote shares instead of totals, an accurate estimate is harder to produce for that year and previous years.
So what might we expect with 2014's popular vote? Given the wide range of the current generic ballot polling, anything from a bare Democratic win to a modest Republican win is plausible, but given the current 2.2 percent Republican lead and historical pattern of the president's party losing seats, as well as which districts are currently competitive, we can likely expect Republicans to narrowly come out ahead, but probably underperform the polling average again with a 2010-sized win being very unlikely.
One thing aiding Republicans is unsurprisingly the uncontested districts. While the distribution between the parties is nearly even this year, just like in 2012 those seats without a Republican on the ballot are more hostile to that party than the ones without a Democrat are to the Democrats. If we repeat the earlier exercise of simply ignoring the presidential loser's vote in every district they didn't field a candidate, Obama's national margin shrinks by 0.6 percent, a smaller amount than other years, but helpful to Republicans all the same.
Now that we've seen how uncontested seats can distort the popular vote, some might ask: "Why does the congressional popular vote even matter when we don't have proportional representation and instead use first-past-the-post, single-member districts?" The popular vote matters because everyone's vote should count equally as a matter of principle and we need look no further than the 2000 presidential election to see many Americans outraged when one candidate carried the popular vote, but another the electoral college and was thus the winner.
Although few pundits were anticipating such an outcome, 2012 saw Democrats win the U.S. House popular vote while failing to win overall control. Some observers, including myself, believe this could be mostly or entirely caused by gerrymandering, while others argued that gerrymandering alone didn't matter and single-member districts were instead to blame. Regardless of the cause, the outcome should simply be unacceptable to a society that prides itself on being free and having equality before the law. If the majority of the electorate consistently cannot win control of a legislative body, it simply lacks democratic legitimacy and must be changed.
Unfortunately this electoral split looks likely to repeat itself, if not in 2014 then in subsequent cycles, as the issues with redistricting whereby Democratic votes are inefficiently more geographically concentrated than Republicans will remain. Furthermore, state legislatures whose members are elected under current Republican gerrymanders will get to control the next redistricting cycle in many states while Republican incumbency advantages from being in the majority will persist.
However it doesn't have to be that way. Voters can push for policies that attempt to thwart gerrymandering as they did in California in 2010 when voters took redistricting out of the hands of the legislature and created an independent citizens' redistricting commission. Similarly, Arizona voters in 2000 created a bipartisan commission tasked with creating fair, competitive districts which has helped to negate the pro-Republican geography bias.
Ideally we would adopt a mixed-member proportional representation system similar to New Zealand's, implemented after multiple elections where the Labour party there won the popular vote yet stayed in the minority. Under such a system, some members are elected by geographical districts and others by party list so that legislative representation is generally proportional to party vote share. However, when we live in a country where a large number of voters either don't know or don't care who controls Congress and likely have no idea the party in the minority won the popular vote, the first step to solving the problem involves informing them one exists at all.
