Dana-Farber Cancer Institute Deploys GPT-4 Institute-Wide In 6 Months

Dana-Farber Cancer Institute, known for groundbreaking cancer research, is also an AI Healthcare pioneer. In 2023, Dana-Farber became the first academic medical center to deploy a LLM for general use. In just six months, they deployed GPT-4 to their 12,500 member workforce. They launched, implemented, and created training material for their workforce to use GPT-4 in a compliant, auditable, and secure way. They are using GPT-4 to streamline work and conduct research, but not in direct clinical care. Although generative AI clinical pilot studies are taking place in many hospitals, Dana-Farber was the first to deploy a LLM for general use in an academic medical center or hospital.

Renato Umeton, Ph.D. and his colleagues recently published a paper in NEJM AI sharing the approach, codebase, and lessons learned during the deployment to help others accelerate their AI programs. The paper entitled GPT-4 in a Cancer Center- Institute-Wide Deployment Challenges and Lessons Learned describes the use of GPT-4 in all business areas, including basic research, clinical research, and operations.

"Clinical AI is both high-ROI and high-risk. By focusing on AI for research and operations we are pursuing use cases that are high-ROI and lower-risk."

Renato Umeton, PhD, Director, Artificial Intelligence Operations and Data Science Services, Dana-Farber Cancer Institute

Case Study Highlights

1. GPT4DFCI was designed with a ChatGPT-like user interface.
2. GPT4DFCI is private, secure, HIPAA-compliant, and HIPAA-auditable.
3. The system is deployed within the Dana-Farber digital premises, so all operations, prompts, and responses occur inside a private network.
4. GPT4DFCI can handle up to 265 text pages per session.
5. GPT4DFCI was initially rolled out to a small group of advanced users, then gradually extended to more employees via a ticketing system starting with a large computational research department and an operations department.
6. GPT4DFCI was socialized on Microsoft Teams Channels and email broadcasts, with training materials that included: a glossary of AI terms, practical suggestions for better prompt engineering, usage guidelines, and best practices.
7. GPT4DFCI is not used in clinical care outside of controlled studies.
8. GPT4DFCI is used to increase efficiency in operational, research, and administrative tasks.
9. During the initial rollout the primary work areas reported were Operations (75%), Research (28%), and Clinical Care (15%).
10. The most reported primary uses of AI were “Extracting or searching for information in notes, reports, or other documents” and “Answering general knowledge questions,” each of which was reported by more than 50% of respondents.
11. The top concern, reported by 65% of respondents, was “Inaccurate or false output,” followed by “Ethics and/or compliance with policies,” at 51%.
12. In addition to ethical considerations, a potential concern for widespread research use is cost. A single query of maximum input and output length in GPT-4 costs about $1. Cost is expected to become a major consideration when they begin enabling much higher-throughput research use cases.
13. Azure teams and Azure OpenAI Service teams from Microsoft supported the application development and testing, together with dozens of internal stakeholders.

References

GPT-4 in a Cancer Center — Institute-Wide Deployment Challenges and Lessons Learned, March 28, 2024, NEJM AI.

Link to paper: https://ai.nejm.org/stoken/default+domain/MBGFT6KIUT9AYKQNJB5Q/full?redirectUri=/doi/full/10.1056/AIcs2300191

The source code, infrastructure-as-code, and training material is available at https://github.com/Dana-Farber-AIOS/GPT4DFCI

• Please see the comments section below for clickable links to the paper in NEJM AI and materials on GitHub

The members of the Dana-Farber Generative AI Governance Committee: Melissa Anderson, Esq.; Kiley M. Belliveau, Esq.; Holly B. Berteau, M.B.A.; Emy C. Chen, Ph.D.; Kevin Clancy, M.B.A.; Michelle Cox, B.A.; Jason M. Johnson, Ph.D.; Sarah Kadish, S.M.; Domenic Leco, M.B.A., Esq.; Naomi Lenane, B.A.; Charlotta Lindvall, M.D., Ph.D.; Kelly Maxwell, Esq.; Anshul Mehra, Esq., M.S.; John Methot, M.S.; Niren Sirohi, Ph.D., M.B.A.; Edward Testa, B.S.; Mary Tolikas, Ph.D., M.B.A.; Mark Tomilson; Renato Umeton, Ph.D.; Eliezer M. Van Allen, M.D.; Jennifer Willcox, Esq.; and Erica Woulf, M.B.A.

Subscribe, Comment, Join Group

I'm interested in your feedback - please leave your comments.

To subscribe to the AI in Healthcare Milestones newsletter click here .

To join the AI in Healthcare Milestones Group click here .

Copyright ? 2024 Margaretta Colangelo. All Rights Reserved.

This article was written by Margaretta Colangelo. Margaretta is a leading AI analyst who tracks significant milestones in AI in healthcare. She consults with AI healthcare companies and writes about some of the companies she consults with. Margaretta serves on the advisory board of the AI Precision Health Institute at the University of Hawai?i?Cancer Center @realmargaretta