Journal Club no.10: Vaccine miracles cancellation
Residual confounding in observational studies – Part II
In the Nonspecific Vaccine Effects Journal Club no. 9 we learnt that a very tightly adjusted statistical analysis in an observational study of the MMR vaccine could not mitigate residual confounding. We also learnt that the impact of confounding by indication and healthy vaccinee bias in observational studies assessing effectiveness of the influenza vaccine are manifest despite adjustment.
??Meaning that although the researcher was well aware of the risk of introducing bias to the data because of confounding and he/ she took ample measures to adjust for it, residual confounding could not be removed.
Why it that important? Because vaccines are continuously being investigated for having miraculous effects on off-target conditions. And it is indeed a captivating thought that is letting on researchers to misinterpret their own observational data in vaccine studies and as we are about to see also letting on reviewers and editors in high-ranking scientific journals. Just take a look at this study: Lancet Public Health 2022; 7: e914–22 with the title: Association between influenza vaccination and risk of stroke in Alberta, Canada: a population-based study.
It almost sounds too good to be true doesn’t it? The influenza vaccine allegedly protects against stroke...?
What are we dealing with here? It is magic ?? ?? ? It is a nonspecific vaccine effect ? Is it a miracle ?? ???
Or is it researchers, who did not pay any attention to what was so clearly pointed out seven years earlier in this insightful paper about…. (yes, you’ve guessed it??) the influenza vaccine ??("italics" represent a quote from the BMC Infectious Diseases paper):
“Both confounding by indication and healthy vaccinee bias are likely to operate simultaneously in observational studies on influenza VE (vaccine effectiveness /red.). Although adjustment can correct for confounding by indication to some extent, the resulting estimates are still prone to healthy vaccinee bias, at least as long as unspecific outcomes like all-cause mortality are used. Therefore, cohort studies using administrative data bases with unspecific outcomes should no longer be used to measure the effects of influenza vaccination.”
The point is: Scientific miracles don’t happen ? and if results of observational data in vaccine studies look miraculous it is because they are biased by confounding, including the paper published in Lancet Public Health mentioned above. Hence it is highly unlikely – well biologically implausible that the influenza vaccine protects against stroke – and it truly strikes me as odd that this never raised a flag with any of the reviewers nor the editor during the peer-review for publication in the Lancet Public Health ??
A recently published and indeed brilliant paper is extremely helpful in S-P-E-L-L-I-N-G the problem of confounding in vaccine studies out ??
So that’s what’s on the menu for this #NSEvaccJC no. 10:
Clinical Epidemiology 2024:16 501–512
The study “Confounding and Negative Control Methods in Observational Study of SARS-CoV-2 Vaccine Effectiveness: A Nationwide, Population-Based Danish Health Registry Study” aimed to illustrate confounding in an observational study design by using negative controls.
We’ve seen negative controls being used in two of the studies that were assessed in NSEvacc JC no. 9. Let’s briefly recap: ??A negative control cannot biologically be explained by the effect of the vaccine, in this case the Covid-vaccine, but it can be biased by the same confounding as the primary outcome (the vaccine effect) can.
In this nationwide population-based cohort study two cohorts were included: Danish residents 60–90 years of age matched 1:1 on age and sex: A vaccinated and a non-vaccinated cohort, 61.052 persons who were Covid-vaccinated between 1 March and 1 July 2021 and 61.052 persons who were not vaccinated before 1 July 2021.
The negative control cohorts consisted of persons diagnosed with Covid infection or acute myocardial infarction, stroke, cancer, low energy fracture, or head-trauma. Outcomes were Covid-infection, negative control outcomes (eg, mammography, prostate biopsy, operation for cataract, malignant melanoma, examination of eye and ear), and death. The researchers calculated the adjusted incidence and mortality rate ratios (aIRR and aMRR).
Notably, the researchers tried to ensure that the two groups that are compared were matched. So they adjusted for age at study inclusion, sex, morbidity, number of hospital contacts and number of inpatient hospitalization days in the year before study inclusion, and geographical region.
What did they find? Well, 61.046 non-vaccinated and 61.046 SARS-CoV-2 vaccinated persons were included in the primary risk analysis of the effect of vaccination on infection.
The vaccinated persons had a higher risk of having a positive Covid test…
AND
…they had more mammographies, more prostate biopsies, more examinations due to blood in the stool, more cataract surgery, more diagnosis of skin cancer, more diagnosis of malignant melanoma, and more eye and ear examinations.
In fact, all of the negative outcome measures came out significantly, with a higher risk / chance of having the outcome in question if the person was vaccinated against Covid.
Table 2 was copy-pasted from the article; the red box was inserted by me.
So, if we were true believers of non-specific vaccine effects (which I’m evidently not ??) – this vaccine (in this case the Covid vaccine) supposedly comes with a series of health examinations. Obviously, this is biologically implausible – and therefore not true. It is residual confounding by inherent differences in health seeking behavior between vaccinées and non-vaccinées, which caused the observed differences in diagnoses, examinations, and procedures.
The paper gets even better ??
Table 3 outlineed the risk of death after Covid infection, and on top of that all the negative control outcomes for mortality. Table 3 was copy pasted from the article; the red box was inserted by me.
领英推荐
Again a true believer of nonspecific vaccine effects really has some serious stuff to dig into here: In the adjusted analyses the persons who were vaccinated against Covid had 77% lower risk of dying from Covid compared to unvaccinated persons…
AND ?? ?? ??
52% lower risk of dying from myocardial infarction
50% lower risk of dying from stroke
54% lower risk of dying from cancer
64% lower risk of dying from trauma
57% lower risk of dying from low energy fracture
53% lower risk of dying from head trauma
Oh my – it appears as if the Covid vaccine truly worked like magic ?? ?? – well, even like a miracle ??. This must be a dream coming true for faithful believers of nonspecific vaccine effects… a miracle replicated?
? ERROR – manifest and substantial confounding ?
It is biologically implausible that the vaccine had a nonspecific protection against head trauma, cancer, fracture, stroke etc.
Once again, the negative controls implemented in this study are spelling out this major problem with the observational study design in vaccine studies: They are prone to substantial confounding that cannot be adjusted for. Being vaccinated (or having your child vaccinated) comes with a behavior that employs a series of actions that are not seen in people who are not vaccinated (or not having their children vaccinated). Vaccinated people are typically better educated, have higher income, smoke less, drink less, exercise more… and the list of factors influencing healthy wealthy vaccinee bias goes on.
Now, the current study, Clinical Epidemiology 2024:16 501–512, did not have access to this type of data and this is listed as a limitation to the study. However, there is plenty of literature out there on healthy-wealthy-vaccinee-bias. Just check it out!
??Lesson learned from this NSEvacc JC: The observational study design is particularly problematic in vaccine studies as the data are at strong risk of bias from confounding despite careful adjustment.
The authors of the study put it succinctly like this in their conclusion – and quite frankly I cannot find better words to describe it myself ?? (“italics” mark a quote from the publication):
“We conclude that negative control methods indicate the presence of substantial confounding in observational studies of SARS-CoV-2 vaccine effectiveness, and such studies should therefore be interpreted critically. Our findings indicate a need for randomized vaccine efficacy studies after the emergence of new SARS-CoV-2 variants and the rollout of multiple booster vaccines.”
This concludes the tenth issue of #NSEvaccJournalClub.
Stay tuned for more
For previous issues of the NSEvacc Journal Club:
Managing Director, Unilabs Pharma Service
5 个月Charlotte, brilliant read (again), thank you. It is rather long but I loved every word - but I also got my vaccinations :-)
Consultant: Drug Development, Training, Quality Management, Process Improvements. Grandfather x 3.5!
5 个月I do hope that many researcher invest the time to read through the very thorough work by Charlotte looking into the issue of the so-called unspecific effects of vaccines. It’s a masterpiece of education into how research must be carefully planned, executed and evaluated using the right biostatistics methods and interpreted sensibly. If I was wearing a hat I would lift it for Charlotte’s efforts.