Non-detect microbial detection and enumeration data are not censored data

A recent paper (Bahk & Lee, 2021) addresses the issue of fitting a distribution describing variability of microbial concentrations in food to datasets that include data regarded as censored. This is a pertinent topic that has been and continues to be debated in quantitative microbiology and influenced by a roughly parallel debate in chemistry. Noting that widely used omission and substitution approaches to handling censored data are inappropriate (Helsel, 2006), Bahk & Lee applied previous modelling of microbiological data by authors such as Busschaert et al. (2010) to develop a method based on maximum likelihood for handling censored data. Critically, such approaches apply the right statistical answer to the wrong question because “non-detect microbial detection and enumeration data are fundamentally not censored data and should not be reported or analyzed as such” (Chik et al., 2018).

Many microbial data are commonly perceived as censored because of long-standing practices of reporting non-detects as concentrations below some detection limit (e.g., <1 cfu/mL) rather than reporting raw data. While the practice may be acceptable for reporting, it can lead to misunderstanding when further interpretation is desired. This is because microorganism concentrations in various media are not measured directly, and the mathematical concept of censored data is misapplied when microbial non-detects that are observed counts of zero (or absence in presence–absence tests) are interpreted as exact measurements of concentration that have been censored. Most classical microbiology methods feature estimation of concentrations from observed count or presence–absence data and corresponding sample volumes (or masses) analyzed. This has been central to the inference of concentration from repeated presence–absence analyses using the most probable number method for over a century (e.g., McCrady, 1915).

The raw data observed in the laboratory are a function of unavoidable random error, and this leads to uncertain concentration estimates and a need to quantify this uncertainty (e.g., Emelko et al., 2010). As discussed in Chik et al. (2018), reporting non-detect microbial data as concentrations below some detection limit is (1) sometimes erroneous because concentrations above the detection limit can also yield samples with non-detect results, (2) problematic because it leads to their misinterpretation as censored data, and (3) inconsistent because it implies uncertainty in non-detect results while ignoring uncertainty in all other data. Methods have been developed to analyze microbial data reflecting the link between what is actually observed in the laboratory and the concentrations that are often of interest and to incorporate this information in quantitative microbial risk assessment (e.g., Schmidt & Emelko, 2011; Schmidt et al., 2013). While such methods may entail greater complexity than can be packaged in a ready-to-use spreadsheet tool, they avert the bias that can arise from misinterpreting non-detects as censored data.

FURTHER INFORMATION:

https://www.dhirubhai.net/posts/philip-schmidt-69821720_reporting-and-handling-microbial-non-detects-activity-6708622699452858368-pIKE

REFERENCES:

Bahk, G. and Lee, H., 2021. Microbial-maximum likelihood estimation tool for microbial quantification in food from left-censored data using maximum likelihood estimation for microbial risk assessment. Frontiers in microbiology, 12. https://doi.org/10.3389/fmicb.2021.730733

Chik, A.H.S., Schmidt, P.J. and Emelko, M.B., 2018. Learning something from nothing: the critical importance of rethinking microbial non-detects. Frontiers in microbiology, 9, 2304. https://doi.org/10.3389/fmicb.2018.02304

Emelko, M.B., Schmidt, P.J. and Reilly, P.M., 2010. Particle and microorganism enumeration data: enabling quantitative rigor and judicious interpretation. Environmental science & technology, 44(5), pp.1720-1727. https://doi.org/10.1021/es902382a

Helsel, D.R., 2006. Fabricating data: how substituting values for nondetects can ruin results, and what can be done about it. Chemosphere, 65(11), pp.2434-2439. https://doi.org/10.1016/j.chemosphere.2006.04.051

McCrady, M.H., 1915. The numerical interpretation of fermentation-tube results. The Journal of Infectious Diseases, pp.183-212. https://www.jstor.org/stable/30083495

Schmidt, P.J. and Emelko, M.B., 2011. QMRA and decision-making: are we handling measurement errors associated with pathogen concentration data correctly? Water research, 45(2), pp.427-438. https://doi.org/10.1016/j.watres.2010.08.042

Schmidt, P.J., Pintar, K.D.M., Fazil, A.M., Flemming, C.A., Lanthier, M., Laprade, N., Sunohara, M.D., Simhon, A., Thomas, J.L., Topp, E. and Wilkes, G., 2013. Using Campylobacter spp. and Escherichia coli data and Bayesian microbial risk assessment to examine public health risks in agricultural watersheds under tile drainage management. Water research, 47(10), pp.3255-3272. https://doi.org/10.1016/j.watres.2013.02.002