Evidence versus truth
By 
Andrew Gelman makes an important distinction in this post between the statistical evidence for a proposition and the ultimate truth of the proposition. The reason this distinction is so important is that politicians and other policy makers have to make decisions not just on the basis of statistical probability, but on the basis of moral judgments regarding what ought to follow as a matter of policy, given what’s known about the relevant probabilities.
Gelman discusses this in the context of some statistically flawed arguments made at the very beginning of the Covid epidemic:
Stanford professor Jay Bhattacharya wrote about a covid study from 2020 that he was involved in, which attracted some skepticism at the time . . .What I wrote in my post is that their article “does not provide strong evidence that the rate of people in Santa Clara county exposed by that date was as high as claimed.” But I also wrote, “I’m not saying that the claims in the above-linked paper are wrong . . . The Bendavid et al. study is problematic if it is taken as strong evidence for those particular estimates, but it’s valuable if it’s considered as one piece of information that’s part of a big picture that remains uncertain.” And I clarified, “When I wrote that the authors of the article owe us all an apology, I didn’t mean they owed us an apology for doing the study, I meant they owed us an apology for avoidable errors in the statistical analysis that led to overconfident claims. But, again, let’s not make the opposite mistake of using uncertainty as a way to affirm a null hypothesis.”
Again, this is a really critical distinction: Just because the statistical significance of the evidence for the truth of X may be overstated as a matter of statistical analysis, for reasons that vary from innocent logical and empirical errors to intentional fraud, that does not in itself mean that X is ultimately false. Policy makers have to weigh not just the statistical significance of the available evidence regarding whether X is true, but the social significance of the possibility that X is true, which is quite a different thing.
Gelman makes the morbidly amusing point that Bhattacharya’s complaints about the unreliability of Stanford-affiliated research are all too well grounded in the context of the Covid epidemic in particular:
A few weeks before the above-discussed covid study came out, Stanford got some press when law professor Richard Epstein published something through Stanford’s Hoover Institution predicting that U.S. covid deaths would max out at 500, a prediction he later updated to 5000 (see here for details). I’ve never met Epstein or corresponded with him, but he comes off as quite the asshole, having said this to a magazine interviewer: “But, you want to come at me hard, I am going to come back harder at you. And then if I can’t jam my fingers down your throat, then I am not worth it. . . . But a little bit of respect.” A couple years later, he followed up with some idiotic statements about the covid vaccine. Fine–the guy’s just a law professor, not a health economist or anyone else with relevant expertise here–the point is that Stanford appears to be stuck with him. In Bhattacharya’s words, “Given this history, members of the public could be forgiven if they wonder whether any Stanford research can be trusted.”
The “idiotic statements about the covid vaccine” from Epstein are themselves really something — I somehow missed this particular entry in that bloviating grifter’s oeuvre at the time — in effect arguing that if a vaccine is safe and effective there’s no need for any kind of government mandate regarding its distribution! Apparently because of the magic invisible hand don’t you know.
That Richard Epstein was once such a big deal in legal academia is by itself some pretty strong statistical evidence that legal academia is kind of a joke, although a particularly bad one.
Anyway this seems like a good time to revisit the fact that in January 2022, which is to say many months after an extremely effective Covid vaccine had been freely available to everyone in the USA, 20,000 people per week were dying of Covid in this country, in no small part because of people like Bhattacharya, Epstein, RFK Jr., and oh yeah somebody named Donald Trump.
Bhattacharya by the way has just been named by Trump to lead the National Institutes of Health. A commenter on Gelman’s post notes:
You should read more about this guy before assuming any *hint* of good faith in his arguments/pronouncements. Are you really serious that you haven’t heard the BS he’s been spewing in courtrooms and the media for the past 5 years straight (??) Judges have thrown out his testimony *repeatedly* on issues related to COVID.
The narcissistic injuries he sustained when the medical community reacted in horror to the Great Barrington Declaration (of which he was an architect) and the Santa Clara study have clearly turbocharged his psychopathy. He’s proven himself, repeatedly, to be a complete monster- slipperier than an eel, unfathomably egotistical, and utterly unqualified to be anywhere near the levers of power. An absolute *disgrace* to the medical profession. Many people will die if he’s given any sort of health policy-related leadership role. You don’t owe him any apologies.
That people who were completely wrong about Covid, in ways that produced hundreds of thousands of eminently avoidable deaths in the US alone, are now being put in charge of the national public health system, is going to be a story that future historians will puzzle over, like Caligula’s equine counsel.