WHAT CAN I TRUST VERSUS WHO CAN I TRUST IN SCIENCE.
David Allison
Dean, Distinguished Professor, and Provost Professor at Indiana University Bloomington
WHAT CAN I TRUST VERSUS WHO CAN I TRUST IN SCIENCE: “studies sponsored by industry were significantly more likely to fulfill all four conditions of informativeness than those not sponsored by industry (50.0 vs. 6.0%, …”
?David B. Allison -- Monday, September 5, 2022
?The distinction between activities taken on because one needs the payment received to take them on versus the pursuit of an idea in science solely because of one's own interest goes back at least to the time of Robert Boyle. Boyle purportedly said "Freed from any ambition to leave my heirs rich, I had no need to pursue lucriferous experiments, to which I so much preferred luciferous ones."[1]
My new puppy, Peanut, is spending a great deal of time trying to figure out who she can trust in this new world. That's an important thing for a puppy to figure out.
In science, our trust in evidence should not be about “Who can we trust?”, but “What can we trust?” That is, what merits our trust based upon the scientific methods used to produce it?
"In science, three things matter: the data, the methods used to collect the data (which give them their probative value), and the logic connecting the data and methods to conclusions."[2]
Reasonable skepticism and doubt are essential components of science. “The key lies in the phrases reasonable doubts and unreasonable doubts. We must resist the urge to use ad hominem arguments and instead focus on the scientific evidence.”[3]
One factor that sometimes leads some people to heighten their skepticism and doubt, especially about the rigor, transparency, or usefulness of research, is who funded that research. A common concern is that a financial conflict of interest may lead the research to be less rigorous, less informative, or less transparent.[4]
A new meta-research publication[5] shows that at least on some indicators, the exact opposite may be true.
After an analysis of 125 randomized controlled trials (RCTs), including many from the domain of diabetes, these authors found "Studies sponsored by industry were significantly more likely to fulfill all four conditions of informativeness than those not sponsored by industry (50.0 vs. 6.0%, p-value <0.001)". This difference in proportion is almost an order of magnitude difference.
This suggests that having funding in general and in particular funding from the industry groups that tend to fund clinical research in this domain, increases the rigor and applicability of RCTs.
This information will likely not be all that unexpected to those of us who anecdotally see how certain branches of industry, especially the pharmaceutical industry, are under such great scrutiny and, perhaps partially as a result, maintain a standard of rigor that far exceeds what many of us have witnessed in academia. It is consistent with related observations reported by my group[6],[7],[8],[9]. Dr. Esther Myers and colleagues conducted a similar analysis of nutrition studies[10] years ago and reported “The more specific hypothesis that nutrition-related research reports that received industry funding were of lower quality than those funded by government sources was rejected. …in nutrition-related research, research reports that reported industry funding cannot be assumed to be of lower quality than those funded by government sources.” In the nutrition and obesity space, funding seems to be associated with greater rigor than is no reported funding, without a strong and consistent apparent difference by funding source. Moreover, pharmaceutical studies, at least on some dimensions, seem to be reported more rigorously than are non-pharmaceutical nutrition studies[11].
Clearly funding matters. The large available funding and high scrutiny of certain elements of industry, particularly the pharmaceutical industry, may promote rigor. There is no doubt that there are cases where industry funding is deleterious, and certainly we know of historic debacles (e.g. See "thyroid storm"[12]). But we need to be open-minded about finding the good and reducing the bad.
This new report by Hutchinson et al.[13] will hopefully be mind-opening for those who reflexively are dismissive of industry-funded trials as some research shows can be the case. An RCT has shown that “Physicians discriminate among trials of varying degrees of rigor, but industry sponsorship negatively influences their perception of methodologic quality and reduces their willingness to believe and act on trial findings, independently of the trial's quality.”[14]
Let us exercise our good judgment and focusing on what we actually know about data and rigor of research studies and eschew reflexive supposition.
What do you think?
领英推荐
REFERENCES.
Professor Emerita at Pennington Biomedical Research Center
7 个月PS. Cute puppy!
Professor Emerita at Pennington Biomedical Research Center
7 个月You know, David, one of the greatest changes I have seen in clinical trials of weight management is the emergence of studies with high retention. I remember well the studies from the 90's and early in the 2000's where it was routine that we would see 40% or more "lost to follow-up". This was true in industry-sponsored and in non-commercially-sponsored studies. The change came with DPP and Look AHEAD, NIH sponsored studies where great effort was put into retention. That morphed to industry and now, our industry-sponsored studies of weight management adhere to high standards for retention. These days the "drug studies" are producing outstanding retention results, epitomized by SELECT which had 97% retention over a mean duration 39 months. It's important to remember that our industry colleagues face great scrutiny from regulators and they must design with rigor and execute with precision to meet the regulatory hurdles. Their greatest asset is their credibility and my industry friends are highly protective of that.
Professor and Scientific Director, Center for Alimentary and Metabolic Science at University of California, Davis
2 年An interesting perspective indeed. Thank you David. That bias and pressure doesn't occur in non-industry funded studies is a myth, since there is an innate pressure to "sell" one's latest idea to NIH panels, etc. My experience with industry-funded research has been very positive, and notably several papers emerged from these studies over 20+ years that rigorously confirmed the null hypothesis (instead of outcomes that might have benefited the industry partner!). The dirty little secret in science is that any given field triangulates toward the truth over time, through critical thinking around multiple studies from diverse investigators, regardless of funding source.
Executive Manager Fondation Pierre-Henri Ducret
2 年Critical thinking is the 3rd component of health , non smoking being no 1 and no alcohol no 2
Founder/CEO | Personalized Nutrition Innovator
2 年Great article David Allison. The source of funding can certainly create bias. However, so can very real pressures that independent researchers face like securing funding, promotion and tenure. The key is to have clear guidelines and independent review to mitigate this bias. I think we need to encourage responsible corporate research to support the translation of science into products that can help people lead healthier lives.