Distinguishing Bad Use of Science from Bad Science
David Allison
Dean, Distinguished Professor, and Provost Professor at Indiana University Bloomington
In 1925, Findlay wrote?“The mental outlook and the attitude of the people as a whole towards science must be changed and the scientific habit and a spirit of trust in science must be cultivated…”[1] The importance of trust in science has long been emphasized and that emphasis has seemingly increased in recent years[2]. Working on trust is vital and so too is working on meriting that trust or promoting greater trustworthiness, as I have written previously[3].
?One distinction that I've not seen thoroughly explored is the distinction between bad science and bad use of science. These are sometimes conflated, but in my opinion, these are distinct concepts. When I hear "bad science" I think that the process of generating data, interpreting those data, or communicating about the interpretations of those data have gone wrong in a non-trivial way. However, that is quite distinct from a third party later misusing or mischaracterizing what would otherwise not be considered bad science.
Some examples of bad science include a study in which the wrong data were analyzed[4], a study with mistaken code[5], a study of thymus gland size with bad sampling leading to erroneous conclusions[6], a study with incorrect analyses leading to invalid conclusions[7], et cetera.
However, there are other studies or sets of studies that, while limited and imperfect as virtually all studies are, would not be described as bad science. And yet there are multiple instances where scientific information has been used badly. This includes exaggerating effects, miscalculations, erroneous interpretations, and so on. That much of the use of science in the criminal justice system in the United States is quite poor is largely agreed upon and a very substantial concern. A National Academy of Sciences report relating to this topic was one of the most influential reports in the Academy's history[8]. A podcast[9] describing the purportedly bad science of forensic science is an example of this conflation in my opinion.
Yet listening to the podcast related to this, it is clear that most of what is being called bad science is often the bad use of science in the criminal justice system and not bad generation of data, analysis of data, interpretation of data, or communication about scientific findings from the scientists per se. Notable examples of bad use of science include erroneous calculations in the Dreyfus affair[10], the prohibition by a company of their employees from wearing bicycle helmets as thoughtfully discussed by Dr. Peter Attia[11], the famous misuse of probability calculations by a non-statistician resulting in the unreasonable conviction of a woman for a double murder of her children who apparently more likely died of sudden infant death syndrome than any malfeasance by the mother[12], similar bad use of science in apparently falsely convicting a mother of poisoning her child with antifreeze fluid[13], and so on.
Why does this distinction between bad science and bad use of science matter? It matters because if we do not make this distinction, some persons may mistake the bad use of science for bad science and the bad use of science may therefore reflect poorly on the credibility of scientific research conducted by the scientific community. In my opinion, we should therefore call out bad use of science just as we should firmly and promptly call out bad science when we see it. In doing so, we should make the distinctions.
We should take the methodologic and moral high ground and note that as a scientific community, we embrace our obligation to make science self-correcting[14] and tout that when the scientific community is pointing out errors in our science and trying to get them corrected that this is science working at its best and doing what it should[15]. Pointing out these errors should enhance the credibility of science as a communal process and not detract from it as others have noted[16]. When we call out the bad use of science, we should make that distinction so that the scientific community is not painted with a broad brush and held responsible for the foibles of other parties who misuse our offerings any more so than we would criticize, for example, an auto manufacturer who carefully and safely designs cars for the fact that an individual drives a vehicle while intoxicated and causes a fatal collision.
What do you think?
Is this distinction apt and important?
If so, how can we best bring this more into the public consciousness?
David B. Allison, PhD – 7-29-22
==============================
Photo Credit[17].
领英推荐
[1] Chemistry in the Service of Man By Alexander Findlay · 1925. https://www.google.com/books/edition/Chemistry_in_the_Service_of_Man/v4lEAAAAIAAJ?hl=en&gbpv=0
[3] https://www.dhirubhai.net/feed/update/urn:li:activity:6956114967355875328?updateEntityUrn=urn%3Ali%3Afs_feedUpdate%3A%28V2%2Curn%3Ali%3Aactivity%3A6956114967355875328%29
[4] https://retractionwatch.com/2016/02/08/authors-used-wrong-dataset-in-study-on-shock-therapy-exercise-in-depression/
Distinguished Professor and Jacobson Professor at Iowa State University
2 年Fantastic article Dr. DA. In addition to everything you mention, I think marketing departments are sometimes guilty of "bad use of science" (BUOS) . A company's R&D team could generate high quality data and the "message" from that research is often misconstrued or embellished in advertising material. Hopefully, it innocently stems from trying to distill complex information into easily digested soundbites by non-scientists, rather than an intentional misrepresentation of results. But regardless, the end point is the same.....BUOS that unnecessarily generates confusion and suspicion in the customer.
CMO, Tangelo. Founder: Diet ID; True Health Initiative. Founding Director, Yale-Griffin PRC (1998-2019). Health Journalist. COVID Curmudgeon
2 年Thank you, David; agree entirely. I have referred to the 'bad use' of science- in some of my own columns here at LI among other places- as the weaponization of science, because it is often intentional, and goal-oriented. 'Bad use' might allow for unintentional misapplication, and while that may well occur, I think the more common abuse is willful. Both bad science and bad use of science are hazards, but the former, I think, is more apt to be of innocent intent. The latter is common, far more likely to involve intentional manipulation, and thus- the more pernicious of the two. Appreciate your insights here, forewarning this audience of these related but distinct threats- David