Major social decision making in conditions of extreme uncertainty

This is from John Ioannidis, professor of medicine, of epidemiology and population health, of biomedical data science, and of statistics at Stanford University and co-director of Stanford’s Meta-Research Innovation Center. He makes the critical point that we are still basically flying blind in regard to this pandemic, which makes it extremely difficult to evaluate which steps are worth taking, especially since the efficacy of various measures remains largely unknown:

The data collected so far on how many people are infected and how the epidemic is evolving are utterly unreliable. Given the limited testing to date, some deaths and probably the vast majority of infections due to SARS-CoV-2 are being missed. We don’t know if we are failing to capture infections by a factor of three or 300. Three months after the outbreak emerged, most countries, including the U.S., lack the ability to test a large number of people and no countries have reliable data on the prevalence of the virus in a representative random sample of the general population.

I mentioned this yesterday: we have no idea if there are at this moment 50,000, or 500,000, or even more, people with the virus in the US. Higher numbers are certainly plausible, given the growing evidence that it could have been circulating in the population for three months or more by this point, as well as evidence that many cases, especially in younger people, may feature mild symptoms or even be completely asymptomatic. (Anecdote: I know two people who’ve contracted the virus. One became very ill, but is now recovering. He ended up in an emergency room, but wasn’t admitted to the hospital. This is an extremely fit 55-year-old, who says it was by far the worst illness of his life. The other case involves a man of similar age, who tested positive three times during a six-week quarantine. He has never had any symptoms of illness).

Because we have no baseline infection rate, mortality statistics at this point have very limited meaning. Ioannidis argues that the few statistics we do have may point to considerably less than apocalyptic conclusions:

The one situation where an entire, closed population was tested was the Diamond Princess cruise ship and its quarantine passengers. The case fatality rate there was 1.0%, but this was a largely elderly population, in which the death rate from Covid-19 is much higher.

Projecting the Diamond Princess mortality rate onto the age structure of the U.S. population, the death rate among people infected with Covid-19 would be 0.125%. But since this estimate is based on extremely thin data — there were just seven deaths among the 700 infected passengers and crew — the real death rate could stretch from five times lower (0.025%) to five times higher (0.625%). It is also possible that some of the passengers who were infected might die later, and that tourists may have different frequencies of chronic diseases — a risk factor for worse outcomes with SARS-CoV-2 infection — than the general population. Adding these extra sources of uncertainty, reasonable estimates for the case fatality ratio in the general U.S. population vary from 0.05% to 1%.

That is, from the extremely limited data we have, he argues we could be looking at an eventual case fatality ratio of anywhere from one in two thousand to one in one hundred. What follows from this estimate?

That huge range markedly affects how severe the pandemic is and what should be done. A population-wide case fatality rate of 0.05% is lower than seasonal influenza. If that is the true rate, locking down the world with potentially tremendous social and financial consequences may be totally irrational. It’s like an elephant being attacked by a house cat. Frustrated and trying to avoid the cat, the elephant accidentally jumps off a cliff and dies.

Could the Covid-19 case fatality rate be that low? No, some say, pointing to the high rate in elderly people. However, even some so-called mild or common-cold-type coronaviruses that have been known for decades can have case fatality rates as high as 8% when they infect elderly people in nursing homes. In fact, such “mild” coronaviruses infect tens of millions of people every year, and account for 3% to 11% of those hospitalized in the U.S. with lower respiratory infections each winter.

These “mild” coronaviruses may be implicated in several thousands of deaths every year worldwide, though the vast majority of them are not documented with precise testing. Instead, they are lost as noise among 60 million deaths from various causes every year.

Although successful surveillance systems have long existed for influenza, the disease is confirmed by a laboratory in a tiny minority of cases. In the U.S., for example, so far this season 1,073,976 specimens have been tested and 222,552 (20.7%) have tested positive for influenza. In the same period, the estimated number of influenza-like illnesses is between 36,000,000 and 51,000,000, with an estimated 22,000 to 55,000 flu deaths.

Note the uncertainty about influenza-like illness deaths: a 2.5-fold range, corresponding to tens of thousands of deaths. Every year, some of these deaths are due to influenza and some to other viruses, like common-cold coronaviruses.

This is obviously a complex and delicate topic, but Ioannidis makes a valuable point when he argues that, in a situation of extreme uncertainty created by what remain at this point radically inadequate data, it’s important to consider the broader social effects of interventions that might be both far too draconian, given the actual risks (which again remain largely unknown at this point), and which might well not work anyway:

In the absence of data, prepare-for-the-worst reasoning leads to extreme measures of social distancing and lockdowns. Unfortunately, we do not know if these measures work. School closures, for example, may reduce transmission rates. But they may also backfire if children socialize anyhow, if school closure leads children to spend more time with susceptible elderly family members, if children at home disrupt their parents ability to work, and more. School closures may also diminish the chances of developing herd immunity in an age group that is spared serious disease.

The one thing we do apparently know about this virus is that it does not, except in extremely exceptional circumstances, pose a serious danger to young children or adolescents (Internationally, there are still practically no reports of any deaths among anyone under the age of 20. The situation among young and middle-aged adults is more ambiguous, but it’s worth noting that as of yesterday South Korea had reported two deaths among people younger than 50, out of more than 8,000 confirmed cases. (I’ve seen some commenters here claim that there’s a serious risk of long-term organ damage among younger adults, but I haven’t seen the evidence for this claim).

Ioannidis also says this, which seems very sensible to me at least:

One of the bottom lines is that we don’t know how long social distancing measures and lockdowns can be maintained without major consequences to the economy, society, and mental health. Unpredictable evolutions may ensue, including financial crisis, unrest, civil strife, war, and a meltdown of the social fabric. At a minimum, we need unbiased prevalence and incidence data for the evolving infectious load to guide decision-making.

The counter-argument is that since we don’t yet have that data, we need to proceed on the basis of worst-case scenario assumptions, which can then be ratcheted back if warranted. The problem with this argument is twofold: is locking down normal social and economic activity for months even doable in a typical developed society, and, even if it is, is it worth incurring the enormous social costs of such a lockdown, which will have been undertaken on the basis of hypotheses about incidence, prevalence, and mortality risk that may well turn out to be radically mistaken?

Here in the US we are not even one full week into the (very) partial employment of such measures, and the answers to both question remain extremely uncertain.

. . .A reply to Ioannidis from Harvard epidemiologist Marc Lipsitch. (Thanks to several commenters)