Leveraging LLMs To Generate Clinical Histories For Oncologic Imaging Requisitions

Please join us on Friday March 7, 2025 for a talk entitled "Leveraging Large Language Models to Generate Clinical Histories for Oncologic Imaging Requisitions" presented by Dr. Rajesh Bhayana. Dr Bhayana is an Abdominal Radiologist at University Medical Imaging Toronto (UMIT), Technology Lead for the Joint Department of Medical Imaging (JDMI), and an Assistant Professor of Radiology at the University of Toronto. Dr. Bhayana’s research focuses on the practical application of AI, particularly LLMs, in radiology to directly improve patient care. His work addresses current gaps in clinical care by optimizing LLMs to solve real-world problems. He is the founder of an online learning platform with over 140,000 subscribers, where his lectures have been viewed over 6 million times globally.

This talk is hosted by the AI Precision Health Institute at the University of Hawai'i Cancer Center. The AI PHI Affinity Talks on the first Friday of each month, feature influential speakers from academics and industry presenting novel applications of AI in medicine from their recent publications. This talk will be presented on Zoom and everyone is welcome to attend.

Registration

Date: Friday March 7, 2025

Time 9:00am HST / 11:00am PST / 2:00pm EST

Topic: Leveraging Large Language Models to Generate Clinical Histories for Oncologic Imaging Requisitions

Speaker: Dr. Rajesh Bhayana, University of Toronto

Please click here to register to attend on Zoom

https://aiphi.shepherdresearchlab.org/event/affinity-group-march-2025/

Study Overview

Clinical information improves imaging interpretation, but physician-provided histories on requisitions for oncologic imaging often lack key details. The purpose of this study was to evaluate LLMs for automatically generating clinical histories for oncologic imaging requisitions from clinical notes and compare them with original requisition histories.

?? In this study, LLMs enabled accurate automated clinical histories for oncologic imaging that were markedly more complete than original requisition histories and preferred by radiologists for image interpretation and safety.

Materials and Methods: In total, 207 patients with CT performed at a cancer center from January to November 2023 and with an electronic health record clinical note coinciding with ordering date were randomly selected. A multidisciplinary team informed selection of 10 parameters important for oncologic imaging history, including primary oncologic diagnosis, treatment history, and acute symptoms. Clinical notes were independently reviewed to establish the reference standard regarding presence of each parameter. After prompt engineering with seven patients, GPT-4 (version 0613; OpenAI) was prompted on April 9, 2024, to automatically generate structured clinical histories for the 200 remaining patients. Using the reference standard, LLM extraction performance was calculated (recall, precision, F1 score). LLM-generated and original requisition histories were compared for completeness (proportion including each parameter), and 10 radiologists performed pairwise comparison for quality, preference, and subjective likelihood of harm.

Results: For the 200 LLM-generated histories, GPT-4 performed well, extracting oncologic parameters from clinical notes (F1 = 0.983). Compared with original requisition histories, LLM-generated histories more frequently included parameters critical for radiologist interpretation, including primary oncologic diagnosis (99.5% vs 89% [199 and 178 of 200 histories, respectively]; P < .001), acute or worsening symptoms (15% vs 4% [29 and seven of 200]; P < .001), and relevant surgery (61% vs 12% [122 and 23 of 200]; P < .001). Radiologists preferred LLM-generated histories for imaging interpretation (89% vs 5%, 7% equal; P < .001), indicating they would enable more complete interpretation (86% vs 0%, 15% equal; P < .001) and have a lower likelihood of harm (3% vs 55%, 42% neither; P < .001).

Conclusion: An LLM enabled accurate automated clinical histories for oncologic imaging from clinical notes. Compared with original requisition histories, LLM-generated histories were more complete and were preferred by radiologists for imaging interpretation and perceived safety.

Rajesh Bhayana

Dr. Rajesh Bhayana is an Abdominal Radiologist at University Medical Imaging Toronto (UMIT), Technology Lead for the Joint Department of Medical Imaging (JDMI), and an Assistant Professor of Radiology at the University of Toronto. He completed his residency at the University of Toronto, where he served as Chief Resident, and pursued a clinical fellowship in Abdominal Imaging at Massachusetts General Hospital.

Dr. Bhayana’s research focuses on the practical application of AI, particularly LLMs, in radiology to directly improve patient care. His work addresses current gaps in clinical care by optimizing LLMs to solve real-world problems. This has included projects to automatically categorize pancreatic cancer resectability from reports, identify actionable incidental findings, auto-generate histories for oncologic imaging, auto-protocol imaging studies, and automatically calculate cancer risk scores (i.e. O-RADS MRI, LI-RADS, etc). Dr. Bhayana is a passionate educator. He is the founder of an online learning platform with over 140,000 subscribers, where his lectures have been viewed over 6 million times globally.

AI Precision Health Institute Seminar Series

In 2022 the AI Precision Health Institute Affinity Group Seminar Series was formed to discuss current trends and applications of AI in cancer research and clinical practice. The Affinity group brings together AI researchers in a variety of fields including computer science, engineering, nutrition, epidemiology, and radiology with clinicians and advocates. The goal is to foster collaborative interactions to solve problems in cancer that were thought to be unsolvable a decade ago before the broad use of deep learning and AI in medicine.

Past Seminars In This Seminar Series

Evaluating & Implementing AI: The Need for a Learning Health System Feb 2025

AI Drug Discovery and Development: Perils and Successes, Dec 2024

Causal Inference With Missing Data and Recoverability, Oct 2024

LLMs in Healthcare: A Case Study from Dana-Farber, Sept 2024

Dynamic Prediction Improved Breast Cancer Risk Prediction, Aug 2024

Large Language Models For Biomedical Research, July 2024

What Happens If We Use Synthetic Data Without Any Curation, June 2024

Predictive AI Models - Data Standards In Action, May 2024

AI Powered Dermatology Tools and Consumer Decision Making, April 2024

AI Decodes Waveforms To Help Prevent Sudden Cardiac Death, March 2024

Mitigating Unintended Consequences of AI in Biomedicine, Feb 2024

How To Build Responsible, Safe, Trusted AI For Precision Health, Jan 2024

Robust Interpretability Methods For Large Language Models, Dec 2023

Machine Learning Captures Insights Into Brain Tumor Biology, Nov 2023

Comparing AI Algorithms To Predict 5 Year Breast Cancer Risk, Oct 2023

Disrupting the Indigenous DNA SupplyChain, Sept 2023

AI Based Lab Test Approved To Phenotype, Grade Breast Cancer, July 2023

Trustworthy AI and Clinical Validation In Breast Cancer Imaging, June 2023

AI For Ultrasound For Real-Time Breast Cancer Decision Support, May 2023

Deep Learning To Diagnose Breast Cancer With High Accuracy, April 2023

Precision Oncology: Empowering Radiologists With AI, Jan 2023

Machine Learning For Personalized Cancer Screening, Dec 2022

AI Driven Surgical Robots To Diagnose/Treat Prostate Cancer, Nov 2022

Subscribe, Comment, Join Group

I'm interested in your feedback - please leave your comments.

To subscribe to the AI in Healthcare Milestones newsletter click here.

To join the AI in Healthcare Milestones Group click here.

Copyright ? 2025 Margaretta Colangelo. All Rights Reserved.

This article was written by Margaretta Colangelo. Margaretta is a leading AI analyst who tracks significant milestones in AI in healthcare. She is based in Honolulu and serves on the advisory board of the AI Precision Health Institute at the University of Hawai?i?Cancer Center @realmargaretta