A data wonk kind of diary here but thought it would be worth mentioning just to show how completely unreliable and off the mark the polling “science” has become lately, latest data point being the Republican primary results. To make it clear, not saying here there was any deliberate move by the pollsters to throw off their results—there were many polls in many different Super Tuesday states yesterday and the usual cautions about how primary voting can be hard to predict—but it’s shocking how so extremely off the Tuesday primary polls were. This isn’t just any given poll but the 538 averages in general (Nate Silver always stressed the “8 different polls from at least 3 different pollsters” rule and still doing it) so just putting together the data form multiple polls didn’t help—it was truly garbage in and out here. It isn’t just MOE margins, and 5 or even 10 percent swings might be understandable, but the eventual results were often more like 10 to 20 percent off as a rule, and in some states (most notoriously Vermont), off by more than 30 points.
And the kicker here? In an alarming recurring pattern, the Super Tuesday polling showed a huge systematic error favoring Trump, while his actual margin over Haley turned out to be much lower across the Super Tuesday states. Bringing that up because it’s raising questions about something weird going on to account for these huge differences between what pollsters seem to be finding, and what actual votes are showing, with the polls showing a consistent and very large systematic error, in form of a high (and very false) level of support for Trump that isn’t actually there when the votes are counted. Similar to the way Democratic candidates (and ballot initiatives) have been way overperforming what the polls seem to say. Of course, NYT/Siena being one of the worst since the 2022 mid-terms—with its bullshit prediction of a huge “red wave” in Nov. 2022 (one of the worst misses by any poll in years, in any election) and downplaying abortion, which turned out to be one of the two top issues for voters then, but it’s not the only one. As we’ll see below.
ABC news convenient put up a summary of the 538 polling averages in a table right before the Super Tuesday vote,
abcnews.go.com/…
(Search engines themselves like Google and Yahoo have tables with the state by state primary voting results for comparison) For ex. the 538 polling average for Virginia showed a blowout Trump win a 48.7% margin of victory over Haley. And the result on Super Tuesday? A 28.3% margin (63.1-34.8%)—off by more than 20%. OK, even 538 is allowed to blow it once in a while, it’s just one primary in one state after all. Right? Right? Well no. 538 average on day before primary for Tennessee showed a 68.8% Trump margin of victory—actual margin 57.8% (77.3 to 19.5). Massachusetts polling gave Trump a 36.8% margin. Actual result? 23% (59.9 to 36.8%). The RCP polling average was within the MOE for 3 other states (mid-single digits, swinging in both directions) in NC, CA and TX.
But to summary here, of the 6 Super Tuesday states that met 538 criteria for showing a 538 polling average, half of them were way off by double digits (one by more than 20%) and in all 3 cases, the 538 polling averages showed a terribly inaccurate systematic error bias in favor of Trump, that was way off from the actual primary ballot counts where Trump didn’t perform nearly as well. And again since these are 538 averages of multiple polls, it’s not just one poll that’s off or an outlier—the entire polling mechanism for Super Tuesday (like we saw in 2022 mid-terms and special elections) has been terribly inaccurate, with systematic errors for Republicans (in the general election polls and ballot initiatives) and for Trump (in the primaries).
Oh, and it gets even worse for the polling predictive value from literally the night before Election Day (it is to say, the polls were basically useless). The ABC News article points out there were a bunch of other Super Tuesday states that didn’t have as many polls taken, so didn’t have a 538 average explicitly, but still had recent polls taken leading right up to Super Tuesday. So it would seem, a reasonable snap-shot and guess of how voters would vote in the primary for these states, at least within the margin of error. Right? Nope. The article doesn’t give specific numbers for most of the remaining states, but one sentence does and really stands out with its predictions—“His (Trump’s) lead is smaller, though still substantial, in the most recent surveys of Utah (27 points) and Vermont (30 points).”
Actual results? Trump won the Utah primary by just 13.6%--56.3-42.7%--with the Utah polling off by almost 15%, way beyond the MOE and again with a false showing in favor of Trump. And Vermont? With the apparent 30% Trump lead in the polling? He lost the state primary to Haley by a little over 4%--meaning the primary polling in that state was off by nearly 35%, again showing a false level of support for Trump!
We are of course a reality based and data driven site here on DKos and we’re necessarily tough on ourselves to make sure we have data to back up these sorts of findings. But that’s exactly what we have here, hard data from the actual Super Tuesday results showing exactly what many of us have been suspecting about the fishy polling results—an enormous systematic bias (here clearly documented) of the polling showing a falsely high level of support for Trump.
To the point that the Super Tuesday primary polls were basically dogshit and useless for getting an actual sense of the voting public, in this case even in the more limited (and presumable more easy to poll) cases of state primaries. We all of course know the polls are useless very far out from the elections, but that’s another reason the complete polling breakdowns here are so astonishing—these were polls of Super Tuesday done (or calculated as an average) right before the primaries themselves, when the supposedly high pressure of being credible makes the pollsters finally “go honest” and get more accurate numbers. And they didn’t, they weren’t even close.
Again it’s worth mentioning here, to debunk some of the counter-arguments used to try to downplay how badly the polls have been missing it, at least since the 2022 mid-terms and special elections that year—
1. It wasn’t just individual polls that were off or outlier polls that missed it on Super Tuesday, it was the full polling average in half of the states that 538 provided a polling average for. They were way, way off, not only by double digits but in one of those cases, by more than 20 percent. Sometimes we’ll see a retort in these discussions that even if one poll looks way off in what it’s showing, “we can at least trust the 538 polling average because it averages the results of multiple polls, so the outliers should cancel out and we should arrive at a more accurate number”. No, really, we can’t trust the polling averages either. As shown here, even the averages of multiple polls were completely off, ie. that multiple polls were garbage and averaging them together led to more garbage. And in being garbage, they showed a strong and hard to explain systematic bias giving Trump a much larger lead and level of support than he actually had when the Super Tuesday results were counted.
2. It isn’t just a matter of whether the state primary was open or closed—that systematic error in the Super Tuesday polls, erroneously favoring Trump, showed up in both open and closed primaries, meaning the massive polling failure can’t be explained by for ex. the difficulty of guessing the voter make-up or making likely voter screens in open-primary states. (That of course wouldn’t be very promising for a poll’s general election predictability either, given the open voting there.) Most of the Super Tuesday states did use open primaries, but the Utah GOP primary was closed and Massachusetts GOP primary for ex. was mostly closed (only Republicans and unaffiliated). And those were also two of the states with the biggest polling misses compared to the Super Tuesday results, way outside of the MOE.
3. Again it’s worth stating to make this absolutely clear, with those polls in so many Super Tuesday states being way off from the results, it wasn’t just a random error in either direction--it was a systematic error in all 5 of those states showing a massive, double-digit overestimate of Trump’s support, that did not occur in the other direction. Including again, in 3 states that used 538 averages across multiple polls, provided right before the Super Tuesday primaries. All 5 of those states showed massive advantages for Trump. All 5 were completely off, some by more than 20% and one by more than 30 percent, in case of Vermont which Trump lost to Haley.
As for why this is happening—we have absolutely no idea. Likely voter screens have always been a mess and a source of huge errors for media polling organizations, since ultimately despite the “polling science” claims, the likely voter screening is based largely off guesswork, and that results a lot from biases and prejudices about who votes and doesn’t. We just have no idea what composition of the electorate is going to show up on Election Day (or like in 2020, in early or mail-in voting).
But it does possibly back up what tons of Kossacks have been saying—at least a lot of the US media is desperate to have a 2024 horserace to sell more ads, papers and clicks. And the perception of biased, completely inaccurate polls going to outrageously exaggerated levels in overestimating Trump (and Republicans in general for general elections and ballot initiatives)? It isn’t just some vague sense—it’s being demonstrated repeatedly since the middle of 2022, and going into the Super Tuesday primaries. There is something systematically, terribly wrong in the polling since 2022 when the actual votes are counted, serious enough that the polls are getting more and more useless. And not just because they’re polling so many months ahead of a general election—even on the very day before voting, and for smaller primary voting blocks in this case, the polls get it wrong. Utterly, completely wrong and misleading.
None of this is to say we should get complacent, we’ve talked about this before and we should absolutely not. We should continue to press advantages against the Republicans in every way we can, especially using Tom Suozzi’s very effective strategy of using their self-inflicted wound recently—taking their own very conservative border bill for crass political reasons (so Trump would have the border as a campaign issue)—and seizing border security as a Democratic issue, mercilessly keeping the GOP’s own-goal here in the news cycle every week. We must continue to support women and girls’ rights to their own bodies, continue to strongly support Ukraine and the right of nations to self defense (especially when we’ve given them security guarantees), continue to address bread and butter issues and speak to voters where they are.
As we’ve talked about before, this absolutely does include Americans’ concerns about rising cost of living and esp housing prices, college and healthcare. To avoid talk of terms like disinflation and GDP--which voters overwhelmingly don’t care one bit about, and that often goes up for all the wrong reasons, like housing bubbles and medical bills. (If anything from canvassing, Americans would be far happier for modest decreases in GDP and prices if it would make things like rent, mortgages and college more affordable) And instead to point out how Trump caused the US inflation struggles with his corrupt handling of the PPP money when covid-19 hit and his gutting of the inspectors office during the pandemic, with all the resulting fraud, that Biden is powerfully confronting with the IRA, better jobs and incomes, the Administration’s tougher prosecutions and better tax collections from billionaires. And, better more balanced interest rate policy, and going after price-gougers and blocks to supply chains. But don’t get worked up by polls, not only due to their prematurity by also by their outrageous level of inaccuracy. There really is something serious and systemtically wrong with them with an unexplained and very systematic error bias in Trump’s direction. And GOTV and work the campaign trail hard, don’t leave anything to chance and work just as hard down ballot.