P-value misuse:

For many years, researchers have used statistical significance tests to answer issues regarding the efficacy of different policy initiatives and interventions. Statistically significant findings were far more likely to be published in journals, covered by the media, and utilized to inform crucial real-world decisions since researchers thought that they were unlikely to be the result of chance.

However, statistical significance is not always what people thought it was. In an unprecedented statement regarding the misuse of statistical significance, the American Statistical Association (ASA) stated in 2016 that policy decisions and scientific conclusions should not be based solely on whether a p-value, which is a measure of statistical significance, crosses a predetermined threshold. For instance, a drug's adverse effects may be disregarded if its p-value is marginally higher than 0.05, which is just below the threshold for statistical significance. Too frequently, statistical significance is mistakenly taken to indicate that "the intervention worked" or "it didn't work." Potential benefits that fall short of the threshold for statistical significance may be overlooked as a result of this excessively dichotomous interpretation of study results.

Going Beyond Statistical Significance: The Bayesian Interpretation of Estimates (BASIE) Framework for Interpreting Results from Impact Evaluations

The fundamental ideas of the Bayesian Interpretation of Estimates (BASIE) evaluation system are outlined in this brief. Without compromising scientific integrity or misinterpreting statistical significance, BASIE assists researchers in evaluating evaluation results. In particular, by contextualizing an impact estimate within the larger body of existing data, assessors can determine the likelihood that an intervention has significant effects. With BASIE, assessors will still offer evidence-based responses to significant policy concerns, but in a more logical manner that is more in line with decision-makers' interests and less prone to misunderstanding. Below is a summary of the five elements that make up the BASIE Framework:

1.?Probability: According to this paradigm, probability is a relative frequency rather than the strength of an individual's views.

2.?Priors: To determine the likelihood that an intervention would have a significant impact, evaluators should consider prior data rather than preconceived notions

3.??Point estimates: The impact assessed using only intervention data as well as the impact calculated using both intervention data and prior evidence (the "shrunken" estimate) should be reported by evaluators.

4.??Interpretation: To determine the likelihood that an intervention had a significant impact, evaluators should utilize past evidence rather than misinterpreting p-values.

5.??Sensitivity analysis: Assessors should determine how much their opinions regarding the effectiveness of an intervention are impacted by the use of various prior evidence. This approach is a crucial method of tackling the difficulties involved in selecting a suitable prior

?

?

?

?