Coming Face-to-Face with Janus (God of Duality)

A New Primer on “Sensitivity, Specificity, PPV, NPV” in Clinical Diagnostics (II)

I mentioned in Part I of this article, "To Tell Heads from Tails (and True from False)...", that my budding interest in math at a young age was initiated after being asked to add up 1 through 10 quickly, without using paper and pencil. The adult who “challenged” me with the math problem told me afterwards that the famous mathematician, Carl Friedrich Gauss (1777 - 1885), used the same strategy to solve a similar problem (summing up 1 through 100) when he was still in elementary school. I did not know whether a young Gauss used paper and pencil for the math problem, but Gauss is generally regarded as one of the greatest mathematicians of all time for his contributions in many fields, including math, physics and geology. So, I thought that was pretty cool at the time.

Now the topic we are focusing on here has to do with the concept of conditional probability (i.e., the probability of A under the condition B, or P(A|B) in mathematical expression). And while conditional probability is one of the most important concepts in probability theory, it could be quite slippery to grasp. Thankfully, Thomas Bayes (1701 – 1761), a statistician, philosopher and Presbyterian minister, who was just before Gauss’ time, gave us the famous Bayes' theorem: P(A|B) = P(B|A)?P(A)/P(B). But instead of trying to grapple further with this slippery concept and potentially losing half of my readers here, I thought I would just try to visualize the PPV value, i.e., P(Disease True if Test Positive) or P(Disease+|Test+), against the prevalence value, i.e., P(Disease True). Here Graph 1 above shows PPV and NPV curves against prevalence for the coin flip test I discussed previously, and Graph 2 shows a case for a much better diagnostic test which has sensitivity and specificity values both equal to 0.950. The graphs provide some interesting insights, and you can check them out (and together with the 2x2 contingency tables in the image for Part I of this article) at your own leisure.

While all the above math may be fun (or not), I now plan to make a more general case of this concept of sensitivity, specificity, PPV and NPV. Let’s say, for the coin flip test, we now switch from a coin that has front and back (or head and tail) to a coin with two faces, and both faces are of equal importance. Here we’ll call for the oracle of Janus, a Roman god with two faces, who can see beginnings and endings, past and future, gates and doorways. And if we are going to roll the dice with Janus to find out which pathway we should take, we will be probably presented with the choices of two doors or more. So, instead of focusing only on one door (Open/Shut, Go/No go), we’d better pay close attention to both Door A and Door B, and we’d better choose which door to open very carefully.

Let’s continue to use UTI as the clinical example here as we are developing a rapid UTI screening and AST product (pls see Velox Q1 2018 Newsletter). For UTI screening, this is supposed to be a “simple” Yes/No or Positive/Negative situation we have discussed before. And at point-of-care, dipsticks are commonly used for UTI screening due to their low cost and simplicity. However, the dipstick tests are not very accurate, leading to false negative rate (FNR) of 6 – 26% and false positive rate (FPR) of 34 – 49%! While I just sneaked in two more diagnostic parameters here, it is pretty obvious that false negatives lead to under-treatment and false positives lead to over-treatment. Furthermore, the current Antibiotic Susceptibility Tests (AST) take 2 – 3 days, and doctors just cannot wait that long to treat patients. Therefore, the very high false positive rate in UTI in conjunction with the lack of a rapid AST has led to the overuse of antibiotics and the continuing surge of antibiotic resistance in UTI. For certain UTI antibiotic drugs, the antibiotic resistance is now already reaching into 70 – 80%! Also, as antibiotics can also cause strong side effects (e.g., resulting in C. diff, acute kidney injury, etc.), doctors are asking for a more accurate UTI screening and rapid AST product that will allow them to (1) diagnose UTI more accurately; (2) choose the right antibiotic drug immediately vs finding out later that the drug does not work.

So, how do the traditional sensitivity, specificity, PPV and NPV numbers apply here? While the UTI screening test is a simple Yes/No situation, for AST, we have two outcomes both of which are important: (a) which antibiotic drug(s) bacteria may be susceptible (S) to? (b) which antibiotic drug(s) bacteria may be resistant (R) to? If we still use the traditional 2x2 contingency table and consider we now have a two-outcome test (S and R) versus the traditional Yes/No test, we could have a new interpretation of the sensitivity, specificity, PPV and NPV numbers for AST results. As shown in tables below, while sensitivity means how sensitive AST can pick up S, specificity in this case really means how sensitive AST can pick up R. And the traditional PPV, NPV values can also be recast in the similar manner. (It should be noted that in the AST product we are developing, the test is run against a panel of antibiotics which should provide doctors more informed choices and help doctors pick the best drug(s) to treat a patient.)

Taking the AST discussion one step further, we can also reconsider the Yes/No screening test as a more general two-outcome (“Positive” and “Negative”) test, and in this case, “specificity” really just means how sensitive the test can identify true negatives. And similar generalization can be made for PPV and NPV. Now that we have gone through the concept of sensitivity, specificity, PPV and NPV from various angles, I hope this is starting to become an “elementary school” math play.

To wrap this up and come back to the urgent problem of rising antibiotic resistance in UTI and other infectious diseases, we are now coming to a point of reckoning; we are now coming face-to-face with Janus, god of duality, in terms of drug effectiveness vs resistance, past vs future, life vs death. And we’d better not throw our dice in a random and haphazard manner.

(Footnote: here is Part III of the article, "In Probability We Trust", which shows how Bayes' Theorem can be applied to compute PPV, NPV, post-test probability from sensitivity, specificity, prevalence or pre-test probability. Part III also serves as a summary. Warning: math alert.)

If you like this post, pls like it here on LinkedIn or share w/ friends!