Journal Club: Misinformation during and after the Covid-pandemic

Nonspecific Vaccine Effects Journal Club no. 5

During the pandemic health care workers (HCWs) taking care of ill patients were at risk of getting infections themselves. The urge to act and to protect HCWs and the willingness to be fooled by an intriguing, yet undocumented story about nonspecific vaccine effects, led to the initiation of a series of randomized controlled trials (RCTs) throughout the world. The RCTs aimed to show that the Bacillus Calmette Guerin (BCG) vaccine protect against Covid in HCW, in the elderly etc.

Note the word story. As previous Nonspecific Vaccine Effects Journal Clubs have shown the story about the #NSEvacc is exactly that – storytelling. Harmless some might say, well let's see about that. This uncorroborated hypothesis led to a series of RCTs including thousands of people. Now these RCTs are gradually being reported. The two most recent RCTs are being assessed in this issue of the #NSEvacc Journal Club in addition to a review of 11 RCTs testing NSEvacc of BCG against Covid.

J Infect Dis. 2023 Sep 29:jiad422. doi: 10.1093/infdis/jiad422. Online ahead of print.

N Engl J Med 2023;388:1582-96. DOI: 10.1056/NEJMoa2212616

J Infect Dis. 2023 Nov 11;228(10):1467-1478

J Infect Dis. 2023 Sep 29:jiad422

The most recently published RCT on nonspecific effects (#NSEvacc) of BCG originates from Denmark. This RCT included 1.221 HCWs from nine Danish hospitals. Participants enrolled into a single-blinded randomized controlled trial and were randomized 1:1 to standard dose BCG or placebo. The primary outcome was days of unplanned absenteeism. Main secondary outcomes were incidence of COVID-19, all-cause hospitalization, and infectious disease episodes.

The results of the main outcome anaylysis was (in the following "italics" reflects copy-pasted text from the publication(s) in question):

“There was no significant effect of BCG on unplanned absenteeism. Mean number of days absent per 1000 workdays was 20 in the BCG group and 17 in the placebo group (RR 1.23, 95% credibility interval: 0.98 to 1.53).”

Table 2 copy-pasted from the publication; the red box was inserted by me and reflects the primary outcome analysis of the RCT

Okay then – no effect was demonstrated on the primary outcome. What about the secondary outcomes then?

“BCG had no effect on incidence of COVID-19 or all-cause hospitalization overall. In secondary analyses BCG re-vaccination was associated with higher COVID-19 incidence (HR 2.47, 95% confidence interval (CI): 1.07 to 5.71), but also reduced risk of hospitalization (HR 0.28, CI: 0.09 to 0.86). The incidence of infectious disease episodes was similar between randomization groups (HR 1.09, CI: 0.96 to 1.24).”

Table 3 and 4 are copy-pasted from the publication – the red boxes are inserted by me.

Table 3 and Table 4 of the publication demonstrate an enormous willingness among the authors to conduct an endless number of statistical tests of any thinkable or unthinkable aspect pertaining to study participants. Some call this exploring the data set. Having observed this from many other RCTs in the topic of NSEvacc the more accurate terminology would be data torture, p-hacking, fishing expedition.

Notably, the numerous statistical tests finally led the data to cave and confess with a few random statistically significant findings (rolling the dice would have been less expensive for society and less inconvenient for study participants), which the authors then spend two full columns of the Discussion to speculate the importance of. Let’s cut things short: This was a null study that clearly demonstrated that the BCG vaccine had no nonspecific effects.

In conclusion: This RCT conducted in an affluent setting in Denmark among 1.221 health care workers showed no nonspecific effect of the BCG vaccine as protection against COVID-19.

N Engl J Med 2023;388:1582-96

In April 2023 the results from an international (Australia, the Netherlands, Spain, the United Kingdom, and Brazil) multicentre RCT were published. The RCT was a double-blinded, placebo-controlled prospective study investigating 3.988 health care workers (HCW). Study participants were followed for 12 months. The RCT had two co-primary outcomes: the incidence of symptomatic Covid-19 and the incidence severe Covid-19 – 6 months after randomization.

Secondary outcomes were ("italics" reflect citations from the publication)

“…the time to onset of Covid-19; the number of Covid-19 episodes; the number of days with symptoms, absent from work, or confined to bed; complications (including pneumonia, receipt of oxygen, hospitalization, admission to a critical care unit, use of mechanical ventilation, or death); and asymptomatic infection.”

The results of the primary outcome outcome analyses (two co-primary endpoints) RCT were that? symptomatic Covid-19 occurred in 132 participants in the BCG group (adjusted estimated risk, 14.7%) and in 106 participants in the placebo group (12.3%) (difference, 2.4 percentage points; 95% confidence interval [CI], ?0.7 to 5.5; P = 0.13); severe Covid-19, as defined in this trial, occurred in 75 participants in the BCG group (7.6%) and in 61 participants in the placebo group (6.5%) (difference, 1.1 percentage points; 95% CI, ?1.2 to 3.5; P = 0.34).

Figure 2 is copy-pasted from the publication; the red boxes are inserted by me and indicate the two co-primary outcomes of the trial.

What does all this mean? It means that neither of the two co-primary outcomes came out statistically significant - implying that the RCT was another null study - I dare say - yet another one.

The secondary outcomes are shown in table 2 of the publication (not shown here), and all of them but a few are not statically significant. However, as a small light in the darkness, in the footnotes of table 2, the authors have made a note, which they should be praised for:

“The widths of the confidence intervals have not been adjusted for multiplicity and may not be used in place of hypothesis testing.”?

The true meaning of this footnote is: Don’t get carried away with a few potentially to very likely random significant findings. They probably mean nothing because we conducted so many statistical tests and we did not account for it!

Hence, the take home message from this large scale international RCT including 3.988 HCWs is that BCG vaccination did not provide unspecific vaccine effects that protected against Covid. This RCT was a null study.

J Infect Dis. 2023 Nov 11;228(10):1467-1478

“If we are uncritical we shall always find what we want; we shall look for, and find, confirmations, and we shall look away from, and not se, whatever might be dangerous to our pet theories.” Karl Popper (1902-1994)

I leave this quote here – because it appears that True Believers of the nonspecific vaccine effects (in this case BCG) repeatedly look away from and will not see the evidence for what it is – a true danger to their pet theory. This has resemblances to religion, hence the terminology True Believers.

In August 2023 a review was published looking into the (at that time) 11 published trials on the BCG-alleged-nonspecific protection against Covid. Looking into the list of authors of this review – they are a striking match to the list of authors of N Engl J Med 2023;388:1582-96 – which has just been assessed above. Irrespective of the fact that the very same group of authors has just in April 2023 published their null study on BCG nonspecific effects in (they call it off-target – but it reflects the same illusion) against Covid in 3.988 HCWs, the review – and I quote:

“…summarizes the results of randomized trials of BCG vaccination for protection against COVID-19 or its impact (“BCG COVID-19 trials”). It discusses potential explanations for the contradictory findings in these and other trials of BCG’s off-target effects.”?

In the definition of the purpose the authors continue:

“These (contradictory findings/ ed.) are addressed in the context of study design, exemplified by the PICOT framework [references 14, 15], namely, those relating to participants, intervention, controls, outcome measures, and time of follow-up.”

What they mean to say by this is:?To all the True Believers out there, do not worry – everything can be explained. Because we will look for and find explanations – and thus it’ll all add up and confirm our ideas in the end.

Let me give you examples e.g. from the Introduction (and I quote):

“BCG vaccination also has beneficial off-target (“nonspecific”) effects on other infections [references 4, 5].… BCG’s off-target effects may be evident only in certain circumstances.”

Well – what a surprise! Nonspecific effects may only be evident in some circumstances. Is that so?! Or perhaps they are only evident if True Believers are assessing the data…?!? That explanation works equally well.

I am confident that Karl Popper would call this an act of uncritical behavior "to look for and find confirmation – and look away from and not see whatever might be dangerous to the pet theories.”

I shall leave it to the reader to dig into the 11 RCTs by yourself. Remember that the Danish RCT is not even included, as this was published after published in September 2023 and the review was published.

All RCTs (but one very small one in 144 patients with type 1 diabetes) point in the same direction:

The BCG vaccine has no nonspecific protective effects against Covid.

A journalistic saying is "Show it - don't tell it." Table 1 both shows and says it all – it’s from the publication – the red box is inserted by me. All results but one is non-significant, i.e. there is no difference between the intervention (BCG) and the control group in whatever outcome measure the trial was investigating.

Now, one would think that the evidence is so overwhelming even to True Believers that they would draw the evident conclusion. Alas, instead readers are in for a tour de force of explanations of why the RCTs are null studies (i.e. "show inconsitent findings" covering

·???????? age

·???????? predisposition/ risk

·???????? prior BCG

·???????? prior Covid-infection

·???????? sample size

·???????? effect size

·???????? power

·???????? BCG strain

·???????? BCG dosing

·???????? other vaccines

·???????? control

·???????? choice of outcome measure

·???????? measurement of severe Covid

·???????? timing

·???????? responders

·???????? and... bias?

I wonder why the weather was not blamed as well, or the day of the week, or the month, or the setting of the moon, menstrual cycles - or not having them, hair colour, liking football or not....

Karl Popper truly nails it, doesn't he:

“If we are uncritical we shall always find what we want; we shall look for, and find, confirmations, and we shall look away from, and not se, whatever might be dangerous to our pet theories.”

Let’s wrap it up. BCG has proven again and again all over the world to have no nonspecific vaccine effects against Covid. True believers seem to be immune to this fact. Some would call it fact resistance. Perhaps this fact resistance is a nonspecific effect that True Believers inadvertently have caught while conducting their endless series of null studies??

Now I joke a bit about it, but the truth is I am truly concerned. Bringing risk and inconvenience to thousands and thousands more of study participants seem not to bother any of the researchers. If it did – they had not initiated the studies in the first place. They would have known the literature - that reflect exactly that nonspecific effects of BCG have never been documented in the primary analysis of clinical trials.

The pecuniary side of the matter is shocking – time, efforts, and money i.e. research resources have been diverted into following an illusion, that no one has been able to document the causative inference of with RCTs for 45 years.

Is there no end to this madness?

The frail hope that anybody has learnt anything is in vain. Because there is many more research resources to be wasted and many more pointless RCTs to come (e.g. Trials. 2023; 24: 636. Published online 2023 Oct 4. doi: 10.1186/s13063-023-07662-w - or surprise yet another trial from Denmark, this time in the elderly: https://www.clinicaltrialsregister.eu/ctr-search/trial/2020-003904-15/DK). According to a review published in March 2021, 31 RCTs were at the time testing if BCGs has “nonspecific effects (NSEs) on the immune system providing protection against unrelated infections.”

Looking at the bright side – we have plenty of content for next year’s Nonspecific Vaccine Effects Journal Club.

This concludes the fifth issue of the #NSEvaccJournalClub.

Stay tuned for more - the Christmas edition is coming up - ho-ho-ho ??

For previous issue of the Nonspecific vaccine effects Journal Club

Issue no. 4

Issue no. 3

Issue no. 2

Issue no. 1