How do you communicate uncertainty without undermining your results?

If the uncertainty is big enough that you fear it will undermine your audience's belief in your data, the problem is the hypothesis, not the presentation of the uncertainty. If something genuinely varies, the variance may be large. Proper controls tell you if your procedure is working, and you may need more precise measuring tools; however, if there is actual variation, that IS the result. If that result is inconsistent with your hypothesis, that's great, your have eliminated one hypothesis (i.e. you have done science). It's time to move on to one of your alternate hypotheses.

Communicating uncertainty without undermining results involves transparency and clarity. I acknowledge uncertainties upfront, presenting them as inherent aspects of the research process rather than weaknesses. I use language that conveys confidence in the findings while also highlighting limitations and potential areas for further investigation. Providing context, such as discussing methodologies and potential sources of error, helps audiences interpret results accurately. Overall, it's about balancing confidence with honesty about the limitations of the study.

Uncertainty refers to the range of possible values or outcomes that are consistent with your data and methods. It can arise from various sources, such as measurement errors, sampling variability, model assumptions, or unknown factors. Understanding and communicating uncertainty is crucial as it reflects the quality and reliability of your results. Acknowledging uncertainty helps you and your audience evaluate the evidence and draw more nuanced conclusions.

Beware of the temptation to use statistics that make your data look or sound better, or that give you statistical significance when more appropriate tests do not. Looks don't matter, correct matters. Data type and distribution determine which descriptive and which inferential statistics are appropriate for quantifying uncertainty. And the statistics determine the sample size, not you. What is up to you and your field are alpha and beta (your tolerance for false negatives and false positives) and the EFFECT SIZE - how big a difference does a treatment have to cause to be scientifically meaningful. After that, it's just math in the form of a power analysis, which online calculators will do for you.

Quantifying uncertainty involves using appropriate methods and metrics that suit your data, objectives, and audience. Common methods include standard errors, confidence intervals, p-values, margins of error, and sensitivity analysis. Choose metrics that follow the standards and conventions of your field to provide a clear and accurate measure of uncertainty. This ensures that your audience can properly interpret the extent and impact of the uncertainty in your findings.

Error bars should always be standard deviation or interquartile distance, or you should show confidence intervals. They should never be standard error. Standard error is not standard. In fact, it doesn't even mean much, it is just one number needed to calculate confidence intervals. Your audience expects the error bars to mean the same thing every time they are used, for example, 67% of the data falls between them. If you use standard error, you are misleading your audience.

Visualizing uncertainty helps convey the magnitude, direction, variability, and precision of your results. Effective visualization techniques include error bars, confidence bands, box plots, histograms, and contour plots. Select visualizations that are clear, accurate, and informative. Consistency in scales, colors, and labels enhances readability and helps prevent confusion or misinterpretation by your audience.

Pay attention to who your audience is. Other people within your discipline or an adjacent one? Use precise language and rely on your statistics to help you tell your story. People outside of your discipline or people who are unfamiliar with research? Use plain language and be able to tell a story about your data. Understand that even using the word 'uncertainty' may cause people to doubt what you have to say, and you may have to work harder to be seen as a trustworthy source of information. Being able to answer questions about your uncertainty like 'what,' 'why,' and 'how' without relying on statistics may be more helpful.

Explaining uncertainty involves providing context and clarity on its sources and implications. Discuss the assumptions, limitations, and potential errors in your methods and models. Use precise and consistent terminology, such as "probable," "likely," "uncertain," or "possible," while avoiding vague terms like "significant" or "substantial." This approach helps your audience understand the nature of the uncertainty and its relevance to your results.

As scientists we are used to communicating uncertainty with other scientists. However when I started talking to business people I found they talk about uncertainty differently and that if I talked about P-values, confidence limits etc they thought I was dismissing the data. We need to understand our audience and how they understand and evaluate/assess uncertainty and where possible communicate in their "language"

Balancing uncertainty means presenting your results with confidence while honestly acknowledging their limitations. Highlight the strengths and contributions of your research alongside the uncertainties. Avoid overstating your findings or making unsupported claims. Emphasize the potential for further research and the areas where your work opens new avenues for investigation. This balance ensures that uncertainty does not overshadow your main findings.

The uncertainty is an important part of your data and therefore of your main finding. It will only overshadow your interpretation if you are drawing the wrong conclusions. Interpret uncertainty and embrace it, it may be telling you something important.

Engaging your audience is crucial for effective communication of uncertainty. Tailor your communication to the background, interests, and goals of your audience. Use clear and concise language, avoiding jargon and overly technical details. Illustrate your points with examples and analogies that make the information accessible. Invite questions and comments, and respond constructively to foster dialogue and understanding.

I wonder if this article illustrates a problem with AI. It doesn't actually correct any misunderstandings or add anything to the conversation, it simply restates what is already out there, presumably because it was trained on what is out there. Since a lot of what is out there is wrong, it just perpetuates whatever there is most of, regardless of quality. That may be fine for some topics, it is not fine for science.

Consider the context in which you are presenting your work. If your audience includes individuals outside your discipline, be prepared to explain uncertainty in more relatable terms. For example, use plain language and storytelling techniques to convey the significance of your findings and the inherent uncertainties. Remember that effective communication is not just about presenting data but also about building trust and credibility with your audience.