We Need Rigor Not Rigor Mortis

John Torous, MD MBI just wrote a good piece on key takeaways from the failures of Mindstrong and Pear entitled "What the recent failures of Mindstrong and Pear tell us about the future of digital mental health" (NOTE: There's a paywall). The piece outlines the companies’ beliefs behind their failures (For Mindstrong, it was “go to market” investor pressures that shot past the development of the evidence base and, for Pear, payor intransigence to pay—even for FDA-approved digital therapeutics). Further, Dr. Torous shares his perspective that the path forward must be an evidence-based one with a rigor that has generally been skipped over in most digital therapeutics. The piece is right on a lot but it opens up a couple of opportunities for further discussion.

Dr. Torous states “This means studies that account for digital placebo effects, are run in real-world clinical settings, involve diverse patients, and can be replicated by outside teams. We already know how to do this. Pharmaceutical companies offer one example of high investment in medical research and development. They invest in thousands of potential drugs and take the top-performing ones through more and more rigorous studies.”?

Dr. Torous’ point is well taken (ie more rigor is generally better) but we collectively have agreed that the entire approach to drug development is fundamentally broken. Brian Finrow 's piece (co-written with Aleks Engel?and?Srinivas Akkaraju) does an excellent job pointing out the the model, as it's currently designed, isn't financially feasible--with an inference that it only worked when capital markets essentially fund start-up biotechs’ development with investors' zero percent government financing via the Fed.?

Should the development of digital therapeutics be pushed down a broken path because it’s the known one? Rigor is critical—rigor mortis isn’t. It’s understandable why we think the path forward should be the same one—it’s painful but it’s proven. Much like the QWERTY keyboard, it doesn’t matter how many attempts you make to build a better model. We value familiarity over efficiency.

A second comment worth discussing: “But that does not mean these products and digital mental health have no future. These failures tell us that patients, clinicians, and payers want digital tools that are effective in real-world settings.”?

Arguably, we have relatively little evidence that people are clamoring for these products. Yes, we’ve collectively put together enough data to “prove” demand but that is often self servicing and not held to statistically meaningful analysis (small, non-diversified, populations with ignored regression to the mean, no valid p values or confidence intervals). There is no group out there at the steps of the FDA demanding new digital tools. What people want is good mental health care that they can afford—the investment and technology communities have decided that a highly scalable digital platform has the margins (with a lower need for specialized labor) and accessibility to provide that care. They just haven’t proved it works yet or people want that modality as their primary form of engagement.?

Digital therapeutics is at a literal crossroads. It is bedeviled by the need to create an evidence base that must be funded and, most investors, digital health specialists or not, aren’t interested in the 30-year payoff that most drugs build an economic model off of. Absent a compelling evidence base with a concomitant economic analysis, payers will keep shrugging their shoulders and let patients bear the burden of overloaded, expensive, specialty networks that only linearly scale at a time when exponential scale is so drastically needed. Further, no digital therapeutic, to my knowledge, proven or otherwise, has demonstrated large scale consumer interest—there is no pink ribbon, ACT UP, or similar group of patients and advocates demanding digital solutions NOW.?

Digital therapeutics has the opportunity to rethink these two problems and break the mold. Rather than following an approval path that fails most investors' timelines and regulators' needs for a rigorous evidence base, let’s reconsider how different paths, like real world evidence, may create a more capital-efficient model to development and commercialization.

To do the first, it requires a chance in thinking on the second topic—where to start. Rather than starting with the experts and working back to the customer, consider starting with the customer and working forward. You don’t need an evidence base to engage a person and you don’t need to promise them an unapproved clinical outcome either. We need to start by creating compelling experiences that people want to engage with, that they want to coalesce around, that they they’re willing to fight for, and that they value with their time and energy if not their money. From there, it becomes a more straightforward question about what endpoints to gather and how to build those endpoints into an evidence base.?

New tools, like LLMs, new approaches (RWE), and new ways of thinking are being brought to life every day. We all agree more pathways for care of all types are critical given a low supply of labor, antiquated educational models, and state-based credentialing and licensing demands that put downward pressure on supply, as well as the fact that there is real demand for better access to convenient, cost-efficient, care paradigms. Mindstrong and Pear aren’t symptoms of a failed need—but they may be symptoms of faulty thinking that tried to walk a path that is designed for far more failures than successes.?