Gen AI Models & AI Optimized to Healthcare

In my introductory article, I introduced a few “pillars” for successful use of Generative AI in Healthcare:

1. Gen AI Models & AI Optimized to Healthcare
2. Compound AI
3. AI Agents
4. Trust & Governance

For the first 3, I want to elaborate a bit on each.? This article is a deeper dive into ‘pillar’ number one: the need for healthcare-optimized AI models. (Subsequent articles will cover pillars 2 & 3, and? I will cover a bit on Trust & Governance in each of these).

With Gen AI in healthcare we’re looking forward to being able to leverage unstructured data from disparate sources across distributed networks to do things like:

• “Get me my medical record”
• “Summarize this chart for pre- and post-encounter use by clinicians and patients”
• “Check this prior auth for compliance, do a medical necessity check, and summarize clinical guidelines against health plan rules”

But it’s not that easy - yet.? It seems rather obvious that off-the-shelf Large Language Models (LLMs) won’t cut it.? It’s the most obvious of my “pillars of successful use of Gen AI in Healthcare” - you need healthcare-specific models or off the shelf models trained or fine-tuned on your data.? It’s worth digging into what can be done to drive successful, trusted use of LLMs in Healthcare, and how to engineer better results (engineering = the people, processes and tools required to build something).??

Generative AI holds immense potential in healthcare, but off-the-shelf Large Language Models (LLMs) alone cannot deliver the necessary value.? Healthcare presents unique challenges — specialized medical language, complex clinical workflows, coding, strict compliance standards, and more — that general-purpose LLMs are not optimized to handle. To overcome these challenges, healthcare organizations will need models and prompts optimized for numerous tasks (and as we’ll see in future articles, you really need compound AI to handle most healthcare workflows). Moreover, ensuring the success of these models requires optimization techniques, from fine-tuning and prompt engineering to robust trust and governance frameworks.

Gen AI for Healthcare Challenges with Model & Prompt Optimization

LLMs have real costs and performance issues.? There are proprietary, fee-based LLMs as well as open source models.? Some Healthcare enterprises have a mix of in-house developed models, external 3rd-party models, and models built into their core applications.? This problem is only going to become more obvious as more models are developed.? Running LLMs at scale comes with significant computational costs and models can also vary in performance time and accuracy, depending on the task at hand. For healthcare organizations, balancing these costs and performance differences with value-driven AI use cases is a challenge. Optimizing models to reduce unnecessary resource consumption, while maintaining accuracy, becomes a critical business need.

Prompt Engineering: To get the best results from healthcare-specific AI models, precise prompt engineering is essential. Even small adjustments in how prompts are phrased can significantly affect the outcome, whether in extracting key data from medical notes or summarizing clinical encounters. Prompts must be continually optimized and tested to ensure that models respond correctly to the nuanced needs of clinicians and administrators. Failing to do so could lead to inaccurate outputs, like misinterpretation of a medical condition or the incorrect extraction of data from clinical notes.

Medical Language & Healthcare Workflow Challenges: General LLMs aren’t ready out of the box to understand the specificity and complexity of medical terminology or the coding and compiance requirements of various administrative and financial processes (like prior auths and claims). Clinical language, for example, is full of nuance, with medical terms, diagnostic codes, and treatment guidelines that are industry-specific. LLMs trained on broad datasets may misinterpret this language, leading to errors in clinical decision-making.? Additionally, healthcare involves intricate workflows that span departments and data sources. LLMs built for general purposes don’t handle these complex processes well.

Compliance Standards: Healthcare organizations must adhere to stringent data privacy laws like HIPAA, which require that AI systems ensure the confidentiality, integrity, and availability of patient data. General-purpose models often lack the necessary mechanisms to meet these regulatory requirements.

Gen AI for Healthcare Need Optimized, Accurate Models, Prompts, Scorecards & More

Healthcare-Specific Datasets for Model Training: Healthcare-specific AI models must be fine-tuned on curated datasets, which may include electronic health records (EHRs), clinical notes, claims data, lab results, prior authorizations, and more - per the specific use case. Individual enterprises often have their own versions of these, so healthcare organizations need a mechanism to train models on their own data sources.? These models must be trained not only to understand medical language but also to navigate the intricacies of clinical workflows, such as patient intake, chart summaries, and care gap identification.

Compliance-Ready Models: Ensuring compliance with data privacy regulations like HIPAA means that models must be trained with privacy protections in mind, such as anonymization techniques and synthetic data, and secure data-handling protocols.? (This can also be a function of architecture, AI Agents and more).

Monitoring Models, Agents, and Workflows with Scorecards:? Scorecards or model scoring can provide a mechanism for tracking the performance of AI models and AI agents in healthcare workflows.? They measure key indicators like accuracy, efficiency, and compliance. Monitoring these metrics ensures that the models are performing as expected across different clinical applications - and in some cases, this information can lead humans-in-the-loop directly to the source data in question to help resolve discrepancies or check model output.? Having data provenance or direct sourcing is a key capability.

Trust & Governance: Trust in AI solutions is paramount in healthcare. Clinicians and administrators need to know how a model arrived at a particular recommendation or outcome. Governance mechanisms must ensure transparency, allowing for clear explanations of model decisions, especially in areas like clinical recommendations or claims adjudication.? Given the strict regulatory environment in healthcare, governance frameworks need to continuously monitor AI models to ensure they remain compliant with evolving regulations. This includes auditing how models handle patient data and making adjustments as needed to maintain compliance

Healthcare-specific LLMs (& AI) Are More Valuable (obviously)

Improved Accuracy and Relevance:? Healthcare-specific models trained on fine-tuned data can interpret clinical notes or healthcare transactions - even faxes and PDFs - with greater accuracy.? These models can extract relevant information from medical records more efficiently, and provide better clinical decision support. This leads to improved patient outcomes, reduced documentation errors, and more reliable AI-driven recommendations.

Cost, Automation Rate & KPI Optimization: Better model performance drive better performance.? If processes are automated - or human-in-the-loop processes augment their work - better model performance will drive higher automation rates, better accuracy, less operator time (humans), lower costs, and numerous downstream key performance indicators (KPIs).

Cost and Resource Efficiency:? Through model optimization techniques, healthcare organizations can significantly reduce the computational resources required for LLM deployment. This not only cuts down on costs but also ensures that AI models can meet the needs of different operational envioronments, for example, near real-time processing times vs. models that may take longer and have different expectations.

Enhanced Care Coordination: By integrating healthcare-specific models into workflows, organizations can streamline care coordination processes. For example, models can help with identifying care gaps and ensuring adherence to treatment protocols, which ultimately improves patient health and organizational efficiency.

Enhanced Trust and Compliance: Healthcare professionals are more likely to adopt AI when they can trust the output, understand the decision-making process, and see the guardrails, for example, how models support human-in-the-loop processes and show the human end-users where data, insights, content and accuracy scores come from (direct data provenance).??

Healthcare-specific AI models are the bedrock of successful Generative AI implementations in the medical field. However, developing, optimizing, and governing these models requires a comprehensive approach that includes performance and cost optimization, prompt engineering, and robust trust and governance mechanisms. By addressing these needs, healthcare organizations can overcome the limitations of general-purpose LLMs and unlock the full potential of AI to improve patient outcomes, reduce costs, and build trust across the industry.

The Autonomize AI Genesis AI platform includes a model hub with the ability to leverage our models plus internally developed or third-party partner models, it helps to enable model evaluation and optimization.

Next, I’ll look at Compound AI - and why “it’s almost always compound AI.”? Why do most healthcare workflows often require multiple models and more? ? Even subsets of workflows like data enrichment, data enhancement, and content creation will require multiple models.? Then we will get into how healthcare organizations can leverage compound AI across the enterprise with AI Agents - or an agentic approach.

Jeffrey Eyestone: I have been in Healthcare Information Technology (HCIT), Payment Processing & Fintech, and Artificial Intelligence (AI) for 25 years.? My Healthcare AI work over the past 7 years has been inside of organizations like the largest payers (United Healthcare, Elevance Health, CVS Health, and numerous BCBS organization), providers (Intermountain Healthcare, MD Anderson), and a number of IDNs (Kaiser Permanente, Ascension).? In my Healthcare AI work I have been responsible for sales and business development, strategy and AI roadmaps, numerous AI use cases (clinical, sales and marketing, customer service, claims and payment integrity, and trust and governance).? I have numerous case studies in these organizations and can share write-ups (challenges > solution > value), methodologies, and best practices upon request.