Evaluating and Implementing AI: The Need for a Learning Health System

On February 7, 2025 the AI Precision Health Institute at the University of Hawai'i Cancer Center kicked off the 2025 season of their popular seminar series. The AI PHI Affinity Talks on the first Friday of each month, feature influential speakers from academics and industry presenting novel applications of AI in medicine from their recent publications. This talk, entitled "Evaluating and Implementing AI: The Need for a Learning Health System", was presented by Robert Gallo, MD from Stanford University. Dr. Gallo discussed his research on the influence of LLMs on diagnostic reasoning in clinical practice and explored findings from a recent randomized clinical trial evaluating the integration of LLMs as diagnostic aids and their potential to enhance collaboration between physicians and artificial intelligence.

Please see recently published papers "Effectiveness of an AI Intervention for Detecting Clinical Deterioration" (Gallo 2024 JAMA internal medicine?184(5): 557-562) and "Large language model influence on diagnostic reasoning: a randomized clinical trial" (Gallo 2024 JAMA Network Open 7(10): e2440969-e2440969.)

Robert Gallo, MD is a medical informatics research fellow in the Department of Health Policy and the VA Palo Alto Health Care System’s Center for Innovation to Implementation, and a Postdoctoral Scholar at Stanford University. His research focuses on the use of informatics tools, including artificial intelligence, to better support physicians and improve patient outcomes. He also has interest in the evaluation and implementation of prediction models. Dr. Gallo’s research focuses on inpatient health services delivery, particularly for diabetes and cardiovascular disease. He obtained his medical degree at Washington University School of Medicine, and subsequently completed his residency training in Internal Medicine at Stanford University.

AI Precision Health Institute Seminar Series

In 2022 the AI Precision Health Institute Affinity Group Seminar Series was formed to discuss current trends and applications of AI in cancer research and clinical practice. The Affinity group brings together AI researchers in a variety of fields including computer science, engineering, nutrition, epidemiology, and radiology with clinicians and advocates. The goal is to foster collaborative interactions to solve problems in cancer that were thought to be unsolvable a decade ago before the broad use of deep learning and AI in medicine.

Past Seminars In This Seminar Series

AI Drug Discovery and Development: Perils and Successes, December 2024

Causal Inference With Missing Data and Recoverability, October 2024

LLMs in Healthcare: A Case Study from Dana-Farber, September 2024

Dynamic Prediction Improved Breast Cancer Risk Prediction, August 2024

Large Language Models For Biomedical Research, July 2024

What Happens If We Use Synthetic Data Without Any Curation, June 2024

Predictive AI Models - Data Standards In Action, May 2024

AI Powered Dermatology Tools and Consumer Decision Making, April 2024

AI Decodes Waveforms To Help Prevent Sudden Cardiac Death, March 2024

Mitigating Unintended Consequences of AI in Biomedicine, February 2024

How To Build Responsible, Safe, Trusted AI For Precision Health, January 2024

Robust Interpretability Methods For Large Language Models, December 2023

Machine Learning Captures Insights Into Brain Tumor Biology, November 2023

Comparing AI Algorithms To Predict 5 Year Breast Cancer Risk, October 2023

Disrupting the Indigenous DNA SupplyChain, September 2023

AI Based Lab Test Approved To Phenotype, Grade Breast Cancer, July 2023

Trustworthy AI and Clinical Validation In Breast Cancer Imaging, June 2023

AI For Ultrasound For Real-Time Breast Cancer Decision Support, May 2023

Deep Learning To Diagnose Breast Cancer With High Accuracy, April 2023

Precision Oncology: Empowering Radiologists With AI, January 2023

Machine Learning For Personalized Cancer Screening, December 2022

AI Driven Surgical Robots To Diagnose/Treat Prostate Cancer, November 2022

Subscribe, Comment, Join Group

I'm interested in your feedback - please leave your comments.

To subscribe to the AI in Healthcare Milestones newsletter click here.

To join the AI in Healthcare Milestones Group click here.

Copyright ? 2025 Margaretta Colangelo. All Rights Reserved.

This article was written by Margaretta Colangelo. Margaretta is a leading AI analyst who tracks significant milestones in AI in healthcare. She is based in Honolulu and serves on the advisory board of the AI Precision Health Institute at the University of Hawai?i?Cancer Center @realmargaretta