If the past is prologue, the polling averages will be off by a few points again this year in many races, but that error may be predictable, at least averaged over many races. And if we could predict the errors, we could do a better job predicting the final margin. Below I show two different predictions for all Senate races not listed as Safe by DKE (except Maine). Please note these predictions are experimental.
Previously, I showed that in close Senate and Governor's races, on average, a tie goes to the Democrat in deep blue states and the Republican in deep red states. We can use a regression based on this and presidential results to predict what the final margin for this year's races will be, in conjunction with current polling averages. In theory, the average error for these regression-based predictions should be less than the average error of the polling averages, but the regression based predictions should still have some substantial misses in some states.
Instead of a regression, we could simply use the past performance of polls in individual states instead. The major drawback to this technique would be lack of data for prior elections. But here's an example of how it works: in Nevada, the regression tells us polls underestimate the Democratic margin by about two points. However, the actual record shows underestimations of one, four, and nine points in elections with a margin of less than 20 points since 2004. This is also confirmed by errors of five and three points in the presidential elections. For Nevada then, we calculate a state-based estimate by adding five points to the Democratic margin. (Details below.) Again, the range is pretty decent, so the final prediction could easily miss by a few points. Hopefully, the average error of the state-based estimate will be less than the average error of the polling average alone.
So here's the current polling averages, and the two different predictions. Using the state-based estimates, for races with polling margins of five points or less, Democrats get a boost in six races, while Republicans are helped in only three.
More below.
Case Study - Hawai'i
The most extreme case in the table above is Hawai'i, where Mazie Hirono's chances move from really good to almost certain. The regression tells us that Hawai'i's basic political structure means the Democratic margin should be underestimated by about seven points.
However, in Hawai'i we definitely need to take local effects into account. The state polling tells us that Hawai'i's state polls are total crap, underestimating the Democratic margin by double digits. Everyone's favorite example is the Rasmussen poll showing Daniel Inouye leading by 13 points in the 2010 senate race; his actual margin of victory was 53 points.
Why is Hawai'i so difficult to poll? Because answering polls is something white people like to do, relatively speaking, and Hawai'i has a large proportion of non-white voters. Meanwhile, nationally at least, of the Asian-Americans and Native Hawaiians who actually do answer a live-caller multilingual poll, in two different examples, about a third will say they're undecided.
This year's Senate elections overall
The most dramatic effect could come in Nevada, where, like in 2010, a slight polling deficit for the Democrat is predicted to move to a slight lead in the election. This is consistent with what we've been hearing about registration and early voting out of Nevada.
A couple points of a boost compared to the polling average is also predicted by both the regression and previous state polls in the close races in CT, WI, OH, and PA. Elizabeth Warren has a boost predicted by both the regression and previous state polls of about 4-5 points, which makes the current situation look very nice for her indeed.
Finally there's the potential heartbreakers - AZ, MT, and VA, three of the closest races, where both the regression and the previous state polls suggest a slight boost for the Republican. However, there is a very important caveat, and that is a lack of previous polling in Montana and Arizona (as well as North Dakota), so the state-based estimate at least is tenuous.
Fine print.
The state-based model was based on averaging the difference between the polling average and the actual outcomes in elections held between 2004 and 2010. All polls from October were included unless a clear trend was apparent, in which case only polls from the final 10 days were included. Elections with third party candidates >5%, recall elections, Research 2000, and Strategic Vision were excluded. In cases where four or five polls were within a reasonable range and one poll had a margin more than 20 points different, the outlier was removed. Presidential elections were not included because of misalignment problems in some states (such as West Virginia); however, in most states, the error for the presidential polling numbers is similar to the errors in Senate and Governor's races.
Races with margins greater than 20 points were not included because of the tendency of voters to jump on the bandwagon in uncompetitive races. On average, races with a margin greater than 20 points had winners outperforming the polling average by about 3-4 points more than would be expected in a more competitive race. Update 10/25/12: I was wrong about that. On average, the blowouts also follow the same regression as the other races, just with much larger errors because the averages are typically of 1-3 polls. Races are now excluded from the averages when they have fewer than five polls. This happens to include most blowouts. If the state has no races with more than five polls, races with fewer than five polls are used in the average.