The WSJ has
a good look, if incomplete, at the best pollsters of 2006. Carl Bialik used the "Mitofsky method" to gauge accuracy -- comparing the point spread in a poll to the final result. Thus if the poll shows a five-point race, and the final result is 10-points, then it's an error of five points.
I looked at five pollsters that were among the most prolific: Rasmussen, SurveyUSA, Zogby (which releases separate telephone and online polls) and Washington, D.C.-based Mason-Dixon. For all but the latter, I used the numbers posted on the organizations' own Web sites. For Mason-Dixon, which keeps some of its poll data behind a subscriber wall, I used Pollster.com to find polls from the two weeks before the election. I checked the results against vote counts as of this Tuesday [...]
There were some interesting trends: Phone polls tended to be better than online surveys, and companies that used recorded voices rather than live humans in their surveys were standouts. Nearly everyone had some big misses, though, such as predicting that races would be too close to call when in fact they were won by healthy margins. Also, I found that being loyal to a particular polling outfit may not be wise. Taking an average of the five most recent polls for a given state, regardless of the author -- a measure compiled by Pollster.com -- yielded a higher accuracy rate than most individual pollsters.
On to the results: In the Senate races, the average error on the margin of victory was tightly bunched for all the phone polls. Rasmussen (25 races) and Mason-Dixon (15) each were off by an average of fewer than four points on the margin. Zogby's phone polls (10) and SurveyUSA (18) each missed by slightly more than four points. Just four of the 68 phone polls missed by 10 points or more, with the widest miss at 18 points.
But the performance of Zogby Interactive, the unit that conducts surveys online, demonstrates the dubious value of judging polls only by whether they pick winners correctly. As Zogby noted in a press release, its online polls identified 18 of 19 Senate winners correctly. But its predictions missed by an average of 8.6 percentage points in those polls -- at least twice the average miss of four other polling operations I examined. Zogby predicted a nine-point win for Democrat Herb Kohl in Wisconsin; he won by 37 points. Democrat Maria Cantwell was expected to win by four points in Washington; she won by 17. (Zogby cooperated with WSJ.com on an online polling project that tracked some Senate and gubernatorial races.)
The picture was similar in the gubernatorial races (where Zogby polled only online, not by phone). Mason-Dixon's average error was under 3.4 points in 14 races. Rasmussen missed by an average of 3.8 points in 30 races; SurveyUSA was off by 4.4 points, on average, in 18 races. But Zogby's online poll missed by an average of 8.3 points, erring on six races by more than 15 points.
So M-D is still tops in the field. Rasmussen again did great (you can see their results matched up against final results at their home page). SUSA did worse than in previous years but still pretty darn good. And Zogby's internet poll was once again a piece of crap.
Looking at more results at this chart of all polling for all races put together by SUSA, we can see that those Majority Report polls were awful. So not all phone robo-polling was great.
But SUSA and Rasmussen have proven themselves again, despite their many detractors.