Make no mistakes about the p-value

The p-value is probably the most discussed, yet utilized and demanded index in Academia. Passable is the definition of index, such that it is used as a diverging tool guiding decision making at the core bottom of important individual and collective processes. Amongst these: drawing conclusions from one’s experiments, applying for those results to be published in a journal and laying foundations for future investigations.

In an ideal world, this is a production chain type of setting which pumps out consistent results, although the same environment who shall produce said results is self- immunizing against the potential harm of their progeny. As scary as it sounds, the mischievous practice behind the resulting chronic condition affecting the academic world is none other than a naive “rush to Gold” of a self-proclaimed flawed machinery. Embodied by the publish or perish ghost, humilis in silence, it achieves disingenuous levels of danger instead as it is given the right to decide true or not, based on numbers.

The same numbers are yet non unanimously interpreted shaping opinions that are not congruent. As for these reasons, the host of the show, the omnipotent p-value is the actual thinking mind directing many scientists across the globe. Be it below the .05 threshold, and its blessing shall descend upon you.

Interesting claims from the lost kids coming back from such a Neverland, who started questioning what is the collateral damage of taking that magic powder and claiming a real discovery, provided such a threshold of .05 [1]. Others instead mistaking the p-value for Pan, instead of seeing it for what it is: a Pied Piper of Hamelin often masked as what some knows being the alpha level of .05 (which comes from a completely different non-Fisherian framework)[2]. Although is it true that ad infinitum (perfect) repetitions of the same experiment in the same exact conditions would eventually lead to the mathematical equivalence of a recursive P — which would then embody the true shape of probabilistic variable —, with the false positives error rate, it is also true that such conditions are a utopia.?

Never has anybody witnessed producing such an enormous mole of perfectly repeated inquiries (i.e. 1000 or 10000 repetitions) — belonging just to Plato’s world of forms —, let alone a single human being capable of discerning a true positive from a false positive, without such omniscient tools at hand.

I personally side with the diffident group of people who decided to give the p-value the benefit of the doubt, having witnessed first hand all the intricacies of it’s gestation, and knowing its own father would trust it only if poised with no other tools at his disposal [3].

References

1. Halsey LG, Curran-Everett D, Vowler SL, Drummond GB. The fickle P value generates irreproducible results. Nature methods. 2015 Mar;12(3):179-85.
2. van Helden J. Confidence intervals are no salvation from the alleged fickleness of the P value. Nature methods. 2016 Aug;13(8):605-6.
3. Ziliak S, McCloskey DN. The cult of statistical significance: How the standard error costs us jobs, justice, and lives. University of Michigan Press; 2008 Feb 19.