Five AI Principles Every MedTech Professional Needs to Know

Understanding AI may seem overwhelming to many MedTech professionals.

In a recent interview with MLV, Etienne Nichols ??? , Senior Marketing Manager, Podcast Host & Community Builder at Greenlight Guru , discussed the current state of AI in MedTech and what every MedTech professional needs to know.

According to Nichols, “We say ‘AI’ as if it's one, scary AI out there (artificial intelligence). And really it's lots of little AIs. And the thing to think about with AIs is that they are helping, but obviously you can never check your brain at the door.”

AI's Transformation of MedTech

The medtech industry is witnessing significant transformation driven by artificial intelligence and machine learning (AI/ML) advancements.

Key among these changes is the FDA's proactive response to the dynamic landscape, evident in its release of a draft guidance titled 'Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence Machine Learning (AI/ML) Software.' Issued on April 3, 2023, this draft guidance outlines navigating regulatory requirements for software classified as a medical device incorporating AI/ML. This shift isn't entirely novel, yet the pace of change is accelerating, prompting the FDA to offer these guiding principles.

The FDA's proactive measures highlight their intent to maintain regulatory rigor in this rapidly evolving sector.

Nichols went on to say, “The [broader] concerns (and you might blend a few different things together to get maybe a more general concern) not necessarily the FDA, but if you zoom out and look at AI in general, one of the biggest concerns is that the data sets that they are learning from are going to be biased one way or another.”?

Intelligence, a human trait we often take for granted, warrants a deeper discussion, especially in the context of medical research.

In designing clinical trials, the focus is now shifting toward fostering diversity in the participant population. Historically, much of the testing was predominantly conducted on white males, leaving a noticeable gap in understanding the impacts on diverse ethnicities and cultural backgrounds. As a result, the industry is currently pushing for broader representation to ensure comprehensive findings that are more universally applicable.

This shift is a positive step towards inclusive medical research.

Nichols added, “Well, the same way we train or we understand our medical devices, a physical medical device, how this medical device is held, how this medical device interacts with this person, we understand that from a clinical trial perspective. Now take that same thinking back over to the artificial intelligence and machine learning.”??

Avoiding Bias

The challenge lies in how medical devices interpret data collected from diverse populations.

When a device gathers extensive data, it's critical to understand what it's learning. Questions arise regarding its performance across diverse parameters—light or dark skin or different finger sizes—and how it adapts its interpretation of this data. The pivotal concern is whether the device's learning process could inadvertently introduce bias against certain demographics.

This scenario underscores the necessity of unbiased learning in medical devices.

Nichols said, “That's the biggest concern right now: How is the artificial intelligence truly becoming intelligent, and is it able to determine whether or not it's becoming biased?”

AI learning patterns and potential biases therein offer a compelling discussion point.

Data sets used for AI training and the "black box" phenomenon, where the decision-making process is opaque, raise questions about possible intentional skewing and bias. When a single person designs a clinical trial or sifts through a data set—an N of one scenario—there's a chance of human bias, despite best intentions. A shift to AI learning from diverse data sets might mitigate this, with each AI entity learning from its unique data pool.

This could result in an intriguing collective AI wisdom conditioned by its learning environment.

Nichols made a great analogy: “So take the historic example that's typically used might be that giant bottle of jelly beans, a massive jar of jelly beans. One person comes, they say, okay, guess how many jelly beans are in there? They might guess way off. Next person comes, [more] wrong guesses. Next person comes, [more] wrong guesses. When you get to a certain point with that crowd, and if you average all their responses, you can actually get pretty close to the answer. And that's kind of the way AI sort of works. It becomes the wisdom of the crowd.”

Training Medical Device Algorithms

Training medical device algorithms requires meticulous attention to the target patient population.

While inclusivity is paramount in clinical research, extensive generalization can be counterproductive for medical devices tailored to a particular indication or patient population. For instance, if your device is designed for individuals aged 60-70 suffering from Alzheimer's in a certain region, training it on this specific demographic is essential. Expanding the training set beyond this could dilute the device's efficacy and relevance.

Narrow-focused training ensures precision and reliability in targeted healthcare solutions.

Nichols states, “If you train it on the entire world, then you're now not specializing for that specific indication. So that's one of the things that you need to be thinking about when you say that black box. Well, the data in is really incredibly valuable because that's going to determine the data out.”

5 Things to Know from the AI Bill of Rights?

The White House put out a guidance that is a blueprint for an AI Bill of Rights.

The White House has introduced an AI Bill of Rights emphasizing five principles: Safe and Effective Systems, Algorithm Discrimination, Data Privacy, Notice and Explanation, Human Alternatives, Consideration, and Fallback.

Nichols went on to summarize these principles:

1. Safe and Effective Systems - This fundamental principle demands that any medical device developed should unequivocally ensure safety and efficacy, a concept deeply ingrained in the consciousness of quality and regulatory professionals.

2. Algorithm Discrimination - This principle calls for careful, unbiased algorithm selection, maintaining objectivity when choosing the data for your machine learning applications.

3. Data Privacy - With the introduction of the Omnibus Act on December 29, 2022, the importance of cybersecurity and data privacy has been underscored. Regardless of your adherence to EUMDR or GDPR, robust data privacy measures are essential in this interconnected digital age.

4. Notice and Explanation - This principle underscores the necessity to clearly communicate and explain the workings of the AI systems to end users, empowering them with understanding and trust.

5. Human Alternatives, Consideration, and Fallback - This principle stresses the importance of having contingency plans or "Plan B" options when the AI system fails or makes an erroneous decision. Safeguarding human intervention or alternatives ensures the system's reliability and adaptability.

If adhered to, these overarching principles provide a comprehensive, principle-based perspective on the guidance and the broader picture.

Navigating Complexity

As we navigate the complexities of AI in MedTech, the insights from Etienne Nichols emphasize the necessity of objective learning and the careful handling of data to prevent biases.?

The White House's AI Bill of Rights provides guiding principles to ensure safety, non-discrimination, privacy, transparency, and human fallback options. A comprehensive understanding of these concepts will aid MedTech professionals in leveraging AI's potential while ensuring the highest standards of ethical practice.

Stay tuned to MedTech Leading Voice for more expert interviews about AI.