Theranos bleeds out...

So what was the ultimate outcome of Theranos' dramatic presentation to the AACC Conference in Philadelphia? Theranos is exiting the direct testing business, shutting down labs and cutting 40% of its workforce. Forbes has called for Theranos to return money to its investors. Indeed, one of the investors, a hedge fund called Partner Fund Management, has already filed a lawsuit alleging that "Among other things, Theranos and its principals knowingly and repeatedly lied that they had developed proprietary technologies that worked, were on the cusp of receiving all necessary regulatory clearances and approvals, and concealed the truth about the commercial viability of their technologies and methods."

Meanwhile, Elizabeth Holmes and Theranos have issued a letter stating they will focus on their new MiniLab platform.

But based on the AACC presentation, is MiniLab actually anything close to the revolutionary transformation in healthcare that Theranos promised?

Here's a photo of some of the MiniLab performance data that Theranos presented at the AACC conference. For the moment, focus on that last row - performance for Potassium. We see the lab reproducibility CV of 3.4% at a level of 2.9 mmol/L and 2.7% at a level of 6.0 mmol/L. Is this imprecision acceptable?

Before we answer that, let's look at the bias for Potassium for a venous plasma sample. The comparison method, we presume, is the conventional Siemens instrumentation that Theranos operated at their labs in parallel to their Edison instruments.

We see that the regression slope is 1.02 with a y-intercept of 0.15. This works out to a bias of 3.2% at a level of 2.9 mmol/L and 0.5% at a level of 6.0 mmol/L. Is this bias acceptable?

While Theranos noted all of this performance was acceptable according to US regulatory guidelines, there is a more objective approach to judging method acceptability, one that provides a more stringent assessment. Using analytical Sigma-metrics, we can rate the performance of these assays on the Six Sigma scale. In fact we can visually plot the performance onto a Sigma Method Decision Chart.

In this graph we note that the lower level of potassium is performing at less than 3 sigma, while the upper level is performing at 4 sigma. This is not an advance in analytical performance, even at the point of care. I suspect there are already POC devices on the market that can outperform this MiniLab potassium method. Most core lab methods are better than this. This means MiniLab may not deliver a revolution in anything - whatever gains they make by miniaturization, etc., will be traded off in losses in precision and accuracy. You may get "a" result from a MiniLab, but you'll have to follow up and confirm "the" result with a real core lab method to be certain.

Most of the methods displayed at AACC had good performance. There was one unusual method, though, and that was HDL. In the first comparison method, HDL had the worst bias, near 9%, which was close to consuming the entire allowable total error budget that Theranos stated was 13%. The source of that HDL goal comes from NCEP references listed at the top right of the slide, however, the legally binding quality goal comes from CLIA, which regulates HDL in the Federal Register.

Total cholesterol, Triglycerides, and LDL cholesterol had very small biases, but the bias on HDL was significant. So Theranos decided to do a more traceable comparison, and compared their MiniLab method (as well as the comparator method) against a NIST standard.

Theranos found that when compared against a reference material from NIST, the MiniLab HDL method was only biased by 3% at both low and high levels, while the comparator method was biased at the lower level by 10% and then at the higher level just 2%. (We don't know that the imprecision of the comparator method is, so we can't do a Sigma-metric calculation.) Suffice it to say, however, that Theranos was basically scapegoating their core lab HDL method (presumably Siemens) for being more biased than their MiniLab HDL method. Not a great customer testimonial.

However, we can say that if the Theranos HDL method has only 3% bias, combined with imprecision of 2.4% at the lower level and 2.5% at the higher level, and applying the CLIA HDL performance goal of 30%, then we can calculate that both methods are better than six sigma.

In this graph, the lighter green dots represent the performance if we use the comparison or bias between the MiniLab and the presumably Siemens core lab instrument. The darker green dots are from the biases calculated against the NIST refererence material. Note that in either case, the method is meeting the CLIA goal of 30% for HDL. While this is great news for the MiniLab method, it's hardly earth-shaking. Many HDL methods on the market right now, in traditional core lab or POC devices, are also performing at world class quality.

The data presented on the MiniLab, as it is currently operating, is not going to revolutionize the laboratory testing market with its analytical performance. After promising a paradigm shift in healthcare, Theranos is now focusing its work on a device that appears to be at best, another "me too" entry into the POC market.