From the beginning of the pandemic, I’ve been skeptical of the model produced by the University of Washington’s Institute for Health Metrics and Evaluation. In fact, I may have gone to pretty good lengths to say that the model was not worth following, simplistic, and consistently wrong. That’s because the model isn’t really a model at all—not in the usual epidemiological sense. It’s just a curve-fitting algorithm that is highly subject to minor changes in the existing dataset.
Really, for anyone looking to get a forecast of where the COVID-19 pandemic is going in the United States, there are a large number of alternatives out there, most of which are on a much better scientific and statistical footing than the IHME projections.
Except now I’m going to take another look at the IHME projection. Because last week it made a horrific change in its numbers, and it’s worth understanding why.
In the first few weeks of the pandemic, the IHME numbers were literally all over the place. When the site was first used in a coronavirus task force meeting, it indicated more than 200,000 Americans would die. A couple of weeks later, the number was 60,000. Not so long after that, it crept up to 80,000. Then 100,000. And for some months it seemed that every time the existing prediction was updated, it would add just enough to stay ahead of the numbers of people who had already died, while never adding enough to account for even a few weeks of real-world data.
As a predictive tool it was absolute crap—and this is more definitely supposed to be a predictive tool. It says so right at the top of the page.
Still, over time the numbers on the projection have become more stable. Which is only to be expected as, horrifically enough, the United States appears to have settled into a plateau where 40,000 new cases and 1,000 deaths in a day is all too typical. Only holidays, when already inadequate testing is further reduced, seem to provide any kind of break. It’s not that hard to project a line whose slope is so sadly unchanging.
So anyone checking in on the top line of the IHME model last week might have been shocked to see that the numbers increased from the previous projection of 317,000 American deaths to 410,000 deaths. What happened?
The biggest thing that happened was simply time. With the Sept. 3 update, the model was extended two more months to cover all the sorry remainder of 2020. But just saying that the numbers got larger because the time period got longer is overlooking some big factors.
First off, in extending the model two more months, IHME is not projecting that things will get better. In fact, two of the three lines they project going forward suggest that things will get significantly worse between now and the end of the year. That’s because while states that fueled the first pool of cases in the United States continue to keep COVID-19 under improving levels of control, at least 10 states—Idaho, Utah, Kansas, Oklahoma, Arkansas, Alabama, Virginia, West Virginia, Illinois, and Iowa—are still in the “ramping up” phase of the outbreak
In addition, since the last time the numbers were adjusted, there have been no motions that even suggest that government, state or federal, is taking more effective action. Instead, business restrictions continued to be lifted, mobility levels continue to increase, and “mask use continues to decline.” On top of everything else, testing rates have plummeted. That rate of new cases is hugely deceptive, because some states are testing at rates that are fractions of where they were at peak, and even smaller fractions of where they should be. Thirteen states had double-digit rates of positive tests. That includes a 20% rate in both Dakotas, and a staggering 90% positive rate for tests in Alabama. Alabama, it appears, has stopped testing anyone except the people who already clearly have COVID-19.
What all of this means is that, horrible as the 190,000 Americans now dead may seem, the IHME model—a model that has consistently undersold the disaster ahead—is now saying that more Americans will die between now and when we can finally throw away the 2020 calendar, than have died through the entire pandemic to this point.
And that’s their good prediction, the one that assumes continued attempts at control by state government, continued masks mandates, and continued social distancing. If those restraints are taken away, the IHME numbers suggest 620,000 dead by Jan. 1.
The IHME model has been pretty terrible at predicting where this thing is going. Let’s just hope that, for the first time all year, their numbers are too high.