A review of the polling in the ten closest Senate races this year shows that automated pollsters are consistently showing worse results for Democrats than those polls conducted by live interviewers. Why would that be? It is not clear, but I have at least two guesses. First, it may be that the enthusiasm gap is leading Republicans (and especially the Tea Partiers) to be more interested in responding to automated polls than Democrats. Second, it may be related to the cell phone bias that appears to favor Republicans (this is assuming that live interviewer polls are more likely to include cell phones than automated polls -- I believe this is true, but am not certain).
Whatever it is, the difference between the results of the two types of polling is striking.
I chose to look at the ten closest Senate races according to the current Pollster.com averages to determine whether there were any significant differences between the results of automated and live interview polls. Those races are:
Illinois: Mark Kirk leads Alexi Giannoulias (40.9% to 40.1%)
West Virginia: John Raese leads Joe Manchin (46.8% to 45.4%)
Nevada: Sharron Angle leads Harry Reid (47.8% to 46.3%)
Pennsylvania: Pat Toomey leads Joe Sestak (45.8% to 44.3%)
Colorado: Ken Buck leads Michael Bennet (46.9% to 45.0%)
Washington: Patty Murray leads Dino Rossi (48.9% to 45.7%)
California: Barbara Boxer leads Carly Fiorina (46.6% to 43.2%)
Kentucky: Rand Paul leads Jack Conway (46.9% to 42.6%)
Wisconsin: Ron Johnson leads Russ Feingold (51.0% to 44.6%)
Missouri: Roy Blunt leads Robin Carnahan (49.7% to 42.2%)
Lets take each of those races one by one:
ILLINOIS
In Illinois, Pollster.com shows Kirk leading Giannoulias by 1.6% among automated polls, while Giannoulias leads Kirk by 0.4% among live interview polls. (I am having difficulty embedding Pollster.com charts, but feel free to filter yourself to confirm). So, according to Pollster.com's filters, automated polling gives Kirk, on average, results that are 2.0 points better than live interview polling. Talking Points Memo Polltracker which has a different sensitivity and thus slightly different numbers (and which thankfully does seem to embed properly) shows an even wider (3.1 point) gap between automated pollsters and live interviewers in Illinois.
Here is TPM's automated poll average for the race:
And the live interview average:
Averaging Pollster.com's and TPM's filter results you find that in Illinois, automated polls favor Kirk by 2.6 points over live interview polls.
WEST VIRGINIA
In West Virginia, Pollster.com shows Raese leading Manchin by 8.0% among automated polls, while Manchin leads Raese by 2.1% among live interview polls. So, Pollster.com shows an astounding 10.1 point advantage for Raese in automated polling compared to live interview polls. TPM shows an even greater (12.2 point) difference.
Here is TPM's automated poll average for the race:
Compared to the live interview average:
Averaging Pollster.com's and TPM's filter results you find that in West Virginia, automated polls favor Raese by 11.2 points over live interview polls.
NEVADA
In Nevada, Pollster.com shows Angle leading Reid by 0.7% among automated polls, while Reid leads Angle by 1.9% among live interview polls. So, Pollster.com shows a 2.6 point advantage for Angle in automated polling compared to live interview polls. TPM, on the other hand, shows no real difference (Angle actually performs 0.1 points better in live interview polls according to the TPM averages).
Here is TPM's automated poll average for the race:
Compared to the live interview average:
Averaging Pollster.com's and TPM's filter results you find that in Nevada, automated polls favor Angle by 1.3 points over live interview polls.
PENNSYLVANIA
In Pennsylvania, Pollster.com shows Toomey leading Sestak by 8.3% among automated polls, while Toomey leads Sestak by 2.8% among live interview polls. So, Pollster.com shows a 5.5 point advantage for Toomey in automated polling compared to live interview polls. TPM shows a smaller, but still significant difference (3.1 points).
Here is TPM's automated poll average for the race:
Compared to the live interview average:
Averaging Pollster.com's and TPM's filter results you find that in Pennsylvania, automated polls favor Toomey by 4.3 points over live interview polls.
COLORADO
In Colorado, Pollster.com shows Buck leading Bennet by 3.0% among automated polls, while Buck leads Bennet by 2.1% among live interview polls. So, Pollster.com shows a 0.9 point advantage for Buck in automated polling compared to live interview polls. TPM shows a similar small difference (0.4 points).
Here is TPM's automated poll average for the race:
Compared to the live interview average:
Averaging Pollster.com's and TPM's filter results you find that in Colorado, automated polls favor Buck by 0.7 points over live interview polls.
WASHINGTON
In Washington, Pollster.com shows Murray leading Rossi by 1.1% among automated polls, while Murray leads Rossi by 6.0% among live interview polls. So, Pollster.com shows a 4.9 point advantage for Rossi in automated polling compared to live interview polls. TPM shows an even slightly larger difference (5.3 points).
Here is TPM's automated poll average for the race:
Compared to the live interview average:
Averaging Pollster.com's and TPM's filter results you find that in Washington, automated polls favor Rossi by 5.1 points over live interview polls.
CALIFORNIA
In California, Pollster.com shows Boxer leading Fiorina by 3.3% among automated polls, while Boxer leads Fiorina by 3.0% among live interview polls. So, Pollster.com actually shows a very slight 0.3 point advantage for Boxerin automated polling compared to live interview polls (CA is the only race of the ten that Pollster.com shows an advantage in automated polling for the Democrat and the difference is tiny). TPM, on the other hand, shows that automated polls slightly favor Fiorina (by 1.2 points).
Here is TPM's automated poll average for the race:
Compared to the live interview average:
Averaging Pollster.com's and TPM's filter results you find that in California, automated polls favor Fiorina by 0.5 points over live interview polls.
KENTUCKY
In Kentucky, Pollster.com shows Paul leading Conway by 8.0% among automated polls, while Paul leads Conway by 2.4% among live interview polls. So, Pollster.com shows a 5.6 point advantage for Paul in automated polling compared to live interview polls. TPM shows a similar large difference (5.8 points).
Here is TPM's automated poll average for the race:
Compared to the live interview average:
Averaging Pollster.com's and TPM's filter results you find that in Kentucky, automated polls favor Paul by 5.7 points over live interview polls.
WISCONSIN
In Wisconsin, Pollster.com shows Johnson leading Feingold by 8.3% among automated polls, while Johnson leads Feingold by 6.3% among live interview polls. So, Pollster.com shows a 2.0 point advantage for Johnson in automated polling compared to live interview polls. TPM shows a much larger difference (5.9 points).
Here is TPM's automated poll average for the race:
Compared to the live interview average:
Averaging Pollster.com's and TPM's filter results you find that in Wisconsin, automated polls favor Johnson by 4.0 points over live interview polls.
MISSOURI
Finally, in Missouri, Pollster.com shows Blunt leading Carnahan by 7.4% among automated polls, while Blunt leads Carnahan by 6.8% among live interview polls. So, Pollster.com shows a 0.6 point advantage for Blunt in automated polling compared to live interview polls. TPM, on the other hand, shows a much larger advantage for Blunt in automated polls (4.3 points).
Here is TPM's automated poll average for the race:
Compared to the live interview average:
Averaging Pollster.com's and TPM's filter results you find that in Missouri, automated polls favor Blunt by 2.5 points over live interview polls.
TAKE AWAYS
So what does all this mean? Well first, a caveat -- there are typically far more automated polls taken of a race than live interview polls. So the sample size for live interview polls may be small in some races (West Virginia is a clear example of this). Still, there are two very noticeable take-aways:
First, in all ten races, automated polls show the Republican performing better than do live interview polls. In California and Colorado, and even Nevada the differences are slight and do not necessarily suggest a consistent pattern. However, in states like West Virginia, Kentucky, and Washington the differences are significant.
Second, on average across the ten races, automated polls show Republicans performing 3.8 points better than do live interview polls. The difference is still a significant 3.0 points if you remove West Virginia as an outlier.
It really does appear that automated polls are consistently showing better results for Republicans than live interview polls. Now, this does not mean that the automated polls are wrong. Perhaps the live interview polls are underreporting Republican support. Still, I suspect that in this environment (where you have a segment of the far right that is very energized and probably enthusiastic about responding to polls), the live interviews (where response rates are higher) are likely to be more accurate. If that is true, it might provide a bit of good news regarding the status of many of these close races. Regardless, Democrats stand in real danger of losing the Senate. There is a lot of work to do in the final days to make sure that does not happen!