Is digital health snake oil?

Some think digital health is snake oil, i.e. a product that is deceptively advertised or promoted to do something that it has not been proven to do or falsely promises to do the job the customer wants it to do. Some think AI is snake oil. Now, people are questioning whether AI is snake oil. What about prescribed digital therapeutics?

As apps are direct-to-consumer health technologies, they represent a new folk medicine. Users adopt these technologies based on trust rather than understanding how they operate. In the absence of medical knowledge and technical understanding of an AIs operations, users are left to look for cues about a technology’s effectiveness. App store ratings and endorsements can replace the expert review of health-care professionals.

Fourteen organizations representing providers, payers, consumer technology companies and employers are teaming up to cut through the noise and raise higher?standards for finding digital health solutions that work and are worth the investment.

Some are fluent in chatbot, turning to a panoply of generative AI models for various tasks throughout the day. Now, chatbots mastering wellness are "snowballing" in companies' benefits enrollment plans, writes The Wall Street Journal. While demand for therapy has surged in the post-pandemic world, supply has failed to keep up. Enter the wellness chatbot. Amazon, for one, offers employees free access to an app that uses AI to "track the moods of users and create a personalized mental-health plan." Supporters say such tools help stave off anxiety and depression, while critics are dubious of their effectiveness, and they raise safety and privacy concerns.

Many digital health companies have a low level of clinical robustness and do not make many claims as measured by regulatory filings, clinical trials, and public data shared online. Companies and customers may benefit from investing in greater clinical validation efforts.

We need to get real about digital health.

In the case of doctors, that means making a QWILT SET.

A key question of interest in this ecosystem remained unanswered up until recently: Is there any scientific evidence that consumer adoption and usage of these wearable devices and mobile health apps actually leads to a tangible change in their behavior, which, in turn, can show up in concrete health care outcomes? It depends.

Scientists from the University of Wisconsin, Madison, Harvard Medical School and Brown University?published a meta-review in?PLOS Digital Health? seeking to examine the efficacy of apps and other mobile health interventions for mental illness.

After examining results from 14 meta-analyses representing 145 randomized controlled trials, researchers said they failed to find convincing evidence of efficacy.

Will Alexa make you take your medicine? Will the Wizard of Wyeth be keeping track?

Will remote patient monitoring improve the quality of outcomes and reduce costs?

What are the issues using mobile applications for postoperative care?

Fitness trackers are used mostly by people who are fit and without chronic diseases. So what? Who cares?

Are you going to let Alexa or Siri decide if you will get pregnant or not?

After all, mobile medical apps don't come with package inserts and that cotton stuffed in the bottle.

Is your home blood pressure monitor reliable and accurate?

Are COVID symptoms checkers accurate and reliable for triage? It is easy to take an off-the-shelf monitoring device, slap a Covid-19 label on it, and tell the world the device can be used to help lift us out of a public health crisis. It is far more difficult to ensure the product can home in on the unique signature of this virus and improve outcomes for patients, especially when it affects people so differently.

For all the cash flooding the sector, though, there’s been relatively little research to show if and how expanded access to telehealth is improving?care during the Covid-19 pandemic.?

Without more comprehensive studies, it’s difficult to know which areas of health care are benefitting from the shift toward more virtual appointments and how much of a difference, if any, it’s making in terms of the quality or cost of care.?

Employee mental health services have become a billion-dollar industry. New hires, once they have found the restrooms and enrolled in 401(k) plans, are presented with a panoply of digital wellness solutions, mindfulness seminars, massage classes, resilience workshops, coaching sessions and sleep apps.

These programs are a point of pride for forward-thinking human resource departments, evidence that employers care about their workers. But a British researcher who analyzed survey responses from 46,336 workers at companies that offered such programs found that people who participated in them were no better off than colleagues who did not.

Like most things, the answer is not straight forward. Consider:

1. Digital health describes categories of products and services that use media, information and communications technologies to exchange medical information with the purpose of adding value to preventing, diagnosing or treating patients with specific medical problems, reducing aggregate and out of pocket costs, improving the health professional and patient experience, providing more equitable access or decreasing administrative waste. Here are the categories that investors love most:

The majority of the 318,000 mHealth apps are general wellness related. Are you stressed this season? Check out this tool that seeks to help improve your mental health.

However, the number of condition management apps are increasing rapidly, now representing 40% of all health-related apps (IQVIA). Only 16% are disease-specific.

Just because one product in one category delivers some user-defined value does not mean the others do as well. There are value gaps.

1. Determining whether a certain product adds value, depends on the category of the product i.e remote sensors, patient-reported outcomes platforms, AI, experience improvers, etc. and the needs and wants of different sick-care stakeholders. Most health professionals want digital health to deliver a QWILT SET. Most patients want convenience and experience improvement. Most policy wonks and politicians want to extend insurance coverage and equitable access. Taxpayers want value and lower out of pocket costs and insurance premiums. Insurance companies and self-insured employers want value. Which of these technologies has achieved the quintuple aim?

1. Most medical apps are designed to be consumer products, not medical devices, and, consequently, do not have to demonstrate safety and efficacy.
2. Most mobile medical apps are not clinically validated to be safe and effective, let alone cost-effective
3. Value is user-defined so whether digital health products do the job the user wants them to do varies.
4. Multiple regulatory agencies are playing catch up creating rules and regulations determining safety, efficacy, and truth in advertising claims
5. Patients and doctors are confused about whether to trust digital interventions
6. Digital health and digital therapeutics ethical issues are pervasive
7. Many digital health interventions have not been demonstrated to improve medical outcomes or reduce their costs to patients and taxpayers footing the bills
8. Business models and reimbursement policies are in continuous evolution
9. ?Cybersecurity and privacy continue to be one of the top challenges plaguing the industry. Do you really want to accept money or rewards for your data?
10. ?Is digital health care becoming a bubble that will soon burst? Where are the profits?
11. Many digital health tools suffer from a less than ideal user experience (UX) and can even create adverse “side effects.” Here is a framework to help you score a digital health solution. The framework consists of?8 principles?under which sit?41 measures?and a?scoring system.

Can we trust digital health? Although well-intentioned, applying trust to AI, and other digital health interventions, is a category error, mistakenly assuming that AI belongs to a category of things that can be trusted. Trust implies entrustment, placing something of value in the responsibility of another in circumstances of vulnerability. The ideal patient-physician relationship reflects this definition. Patients place their health in physicians' voluntary responsibility and discretion and believe in physicians' benevolent motives.?

From text messages to mobile apps, digital health devices are becoming increasingly important in clinical trials for their ability to streamline trials, lower site burden, and improve the patient experience.?However, manufacturers must consider the safety, reliability, and convenience of these devices in order to effectively implement them into medical device trials. The digital components of these medical device trials must adhere to the same rigorous regulatory standards as the device itself, which can pose significant hurdles for some sponsors. Those hurdles include:

• Usability and Accuracy.?Sponsors must be able to determine that a device is providing the desired endpoint values in a trial. The metric should be accurate and presented in a usable format.
• Safety.?The manufacturer should be able to provide highly secure methods for transmitting data between the digital health device and the analysis site.
• Convenience.?The manufacturer should be able to provide logistical support to decrease the site and subject burden. They should also be able to provide full documentation of engineering verification for the devices.
• Ease of Use.?It is important to consider how the patient will interact with the device. It needs to be an appropriate size and weight, and it should allow the patient to move and behave in the same way he or she normally would.
• Reliability.?To maintain data continuity, the device should have a battery life sufficient to allow it to collect data for long periods of time with minimal glitches.
• Engagement. Like prescription drugs, if apps are to work, patients need to use them the way they are intended to be used.
• Attrition. There are no shortages of digital health technologies to help obese patients lose weight and keep the weight off over time. Like most apps, however, most don't make using them a habit so they stop using them.
• The perils and pitfalls of DIY medicine

A recent review concluded?that "Safety of apps is an emerging public health issue. The available evidence shows that apps pose clinical risks to consumers. The involvement of consumers, regulators, and healthcare professionals in development and testing can improve quality. Additionally, mandatory reporting of safety concerns is needed to improve outcomes." The same arguments can be made for aspects of EMRs, telemedicine, remote sensing, AI and several other digital health technologies

A new overview of systematic reviews of published randomized control trials of mobile health apps found just 23 RCTs of currently-available apps have been conducted, and less than half of those showed a positive health effect from the app in question.

A group of researchers from the Centre for Research in Evidence-Based Practice at Bond University in Queensland, Australia conducted the review, which was?published in Nature's new Digital Medicine journal.

Take suicide prevention as an example. Recently published meta-analyses suggest that the existing tools have inadequate reliability and low positive predictive value (PPV) in distinguishing between low and high-risk patients. For instance, the meta-analytically derived PPV of studies based on 53 samples was 5.5% over an average follow-up period of 63 months . The majority of suicides occurred in the patient groups categorized as “low-risk,” as they vastly outnumber the “high-risk” group. Furthermore, most patients in the “high-risk” group did not die by suicide, because of the relative rarity of the outcome. In addition, the 5.5% risk of suicide for “high-risk” patients pertains to a time interval of more than 5 years; from a clinical perspective it is more helpful to identify those likely to die by suicide within much shorter time frames, namely weeks or months. No relationship between the precision of risk assessment tools and date of their publication has been found, suggesting no radical improvement over the past 50 years.

Or. what about sleep apps? Are they garbage?

Will Noom result in lost pounds that stay off?

But Noom is entering the spotlight at an especially charged point in time. The technology sector has received increasing scrutiny for the way its products can cause potential harm. The Wall Street Journal last year published internal research from?Meta Platforms?that showed how Instagram can?promote fixation on dieting and weight, exacerbating eating disorders in some young users.?

Recently, Weight-loss app Noom, known for its psychology-based approach to dieting which is oft the subject of controversy, is downsizing. The $3.7 billion platform laid off 180 coaches on Thursday in what was described as "an evolution to the coaching model," per Business Insider. It's part of a larger decision to lay off at least 25% of its coaching staff – with plans to cut another 315 employees over the next 10 days. The move comes as Noom looks to pivot into video-based coaching. The company has made headlines for investigations into its "non-diet" premise and its misleading ad tactics.

But, there is controversy about how we determine whether digital health products are not only safe and effective, but have other characteristics as well and whether the randomized clinical trial should continue to be the "gold standard"

Here is a suggested way to score digital health products and services based on technical, clinical, cost and user satisfaction metrics.

Finally, we are living in a ecoystem economy. The digital health cyber nervous system is in its early stages. Until and unless there are interoperability and multisystem integration, digital health technologies will fail to deliver their full potential and will be wasteful and potentially harmful creating unintended consequences.

So, is digital health snake oil? It depends. Some are. Some are not. Maybe some day, some apps will require a prescription and others will be over the internet (OTI) Like they say, where you stand depends on where you sit.

If tech companies want their digital health tools to gain market adoption, their products must work as intended. And for that to happen, physicians must be involved in their initial design, testing and prototype development. However, for that to happen, doctors will have to not only put a hand up to help, they will have to show up to help, something most are unwilling or unable to do, since digital health design, development, and deployment requires not just clinical and workflow knowledge, but some technical and business/entrepreneurial knowledge, skills, attitudes, and competencies as well.

But, until then, we have much to do before we can stop asking the questions that patients are answering on You Tube, rather than health service researchers answering in the medical literature. As my colleague, CT Lin explained, "?Over-reliance on technology often leads to unanticipated consequences. The technology adoption curve always starts with “the peak of overinflated expectations” followed soon by “the valley of despair.” We find that we can mitigate that valley by having a thoughtful team of physician informaticists who understand patient care AS WELL AS technology, communication, leadership and change management. These invaluable individuals lead teams that can anticipate and avoid risks to reach our goals."

A more recent article concludes, "Currently, no major clinical guidelines have incorporated digital health tools. Few digital health technologies have been rigorously shown to be equivalent to standard, accepted measurement devices. Clinical trials are ongoing and there is clear interest in determining the full potential of digital health. Thus, given the expense and unknown benefit-risk profiles of many digital health tools, they should, for the moment, be considered experimental."

To address these concerns, the Digital Medicine Society (DiMe) and the Veterans Health Administration (VHA) launched a?value-driven framework?for evaluating healthcare innovations.

The organizations noted that previous frameworks to evaluate the value of healthcare innovations have been limited in focus because of pressures from third-party payers, typically being restricted to analyses of economics, not quality of life; to assessments of population- or individual-level benefits, not both; and to measurements only at a specific time point, not over time.?In contrast, they say, their new evaluation framework seeks to capture both financial and quality-of-life effects, reflects value over various periods of time, gathers perspectives from all stakeholders, and assesses effects on both the individual and the system of care.

Findings on cost-effectiveness of digital interventions showed a growing body of evidence and suggested a generally favorable effect in terms of costs and health outcomes. However, due to the heterogeneity across study methods, the comparison between interventions still remains difficult

A?recent article?from the University of Warwick goes further and asks if technologists should undertake a Hippocratic oath, comparing the issues posed by new technologies to that of the moral question posed in the hit movie Jurassic Park;

“Your scientists were so preoccupied with whether or not they could, they didn’t stop to think if they should.”

Arlen Meyers, MD, MBA is the President and CEO of the Society of Physician Entrepreneurs on Substack and Editor or Digital Health Entrepreneurship