GPT passes medical exams: Breakthrough or hype?

- Recently you may have seen some pretty shocking headlines about AI in healthcare. You know the ones I'm thinking of, right? Stories like GPT-4 passes medical exams or GPT-4 is more empathetic than doctors. This kind of news stirs up a mix of excitement and some skepticism. So let's break down the stories behind the headlines to separate hype from reality. To level set, GPT models are sophisticated generative AI trained in essence to predict the next word given a sequence of words. The ChatGPT version was further trained with reinforcement learning from human feedback to teach the model how to output human-like text responses. The training data for GPT included massive amounts of content from the web, and it was a breakthrough to discover that these very, very large AI models with massive amounts of computational power led to extremely diverse capabilities, including many that it was not specifically designed to do. These capabilities were termed emergent behaviors, and one of them is what appears to be a working knowledge or in technical lingo learned representations of medical concepts. GT four's emergent behaviors in medicine are explored in depth in a book called "The AI Revolution in Medicine: GPT-4 and Beyond". And it talks about how GPT-4 can pass medical exams. Now does that mean that GPT-4 can practice medicine? Well, no. You see, these tests are not reflective of the real practice of medicine. In fact, in my 20 years, I've yet to see a patient show up in my office with a multiple choice list of what might be wrong. But nonetheless, GPT-4's learned representations of real healthcare concepts are extremely useful and they can make healthcare more efficient, accurate, and ultimately more accessible. Okay, now let's talk about the other headline. GPT-4 was found to be more empathetic than human doctors. Now I have to say this one was pretty counterintuitive. We've always assumed that human empathy was irreplaceable, but here's the catch. While GPT-4 offered responses that were perceived as more empathetic, there are a couple things at work here. One is that in this study, GPT-4 generated longer, more complete responses than the busy human physicians. So the patients likely judged these responses as more empathetic, but there's a little bit more to it. Because these models were trained to align with human interaction, they're really good at inferring the context of a chat interaction. This contextual understanding is very similar to how humans perceive empathy, and that's what makes these models so useful and why ChatGPT has hundreds of millions of users. So together these models have useful skills that we can use in healthcare. One is a working understanding of medical concepts like passing medical exams. And the other is a perception of empathy by inferring context. These twin capabilities are already being applied by helping healthcare providers automatically draft medically accurate and empathetic responses to patient messages that then the doctors can edit and quickly sign. This helps them get over that con time consuming blank page or cold start aspect of writing. And this is just one area where these models are poised to make a doctor's life a little bit easier and patient care a little bit sharper. So the bottom line is that there are some real game changing capabilities of generative AI in healthcare behind some of the hype. And this offers a future with efficient, more accessible, yet still deeply human healthcare.