Getting value from GenAI, with Noelle Russell

We may have reached the top of the hype cycle for ChatGPT and other GenAI. After all of the breathlessness, pundits are starting to question its value. Both Ezra Klein and Kevin Rouse confessed they were underwhelmed when they asked ChatGPT to help them write their books. It’s not surprising but not the point.

As a standalone computer app most people won’t find ChatGPT all that valuable, but when it is knit into a larger software program or ecosystem it makes everything easier to use and more useful, aka: a copilot.

The first time I joined a Teams meeting late and the copilot offered me a high-quality summary of the conversation my jaw dropped.

Epic and Nuance are partnering on software to listen in on a doctor-patient conversation – with permission! – and use GenAI to analyze the recording, draft the doctor’s notes, and suggest updates to the electronic patient record. The goal is to let the physician pay full attention to the patient rather than splitting attention with a computer screen.?

Other organizations are using GenAI to read customer emails and enter the issues into their systems as a request for service, a product problem, or something else. GenAI is handling simple requests saving their human service reps to handle complex cases.

The fastest path to getting business value from generative AI is to add it to an existing computer system to reduce low value work. My friend Noelle Russell knows this better than pretty much anyone. As a Microsoft AI MVP she coaches organizations in how to identify and create tangible value with GenAI.

In May she walked me through her process, short handing it into three steps: boardroom, white board, keyboard.

Boardroom: Work with senior leadership to define a small number of big pain points in current processes. Ideal targets are long call or wait times in a contact center because it’s an expensive problem and you can easily measure and assign value to an improvement. ?For her first engagement with a client Noelle recommends picking a small project with crisp metrics and leaving the hard-to-measure challenges for after she has established credibility.

White-board: Work with the technical and business owners of the system to identify steps in the process where adding GenAI can help. In contact centers common solutions include having AI handle more routine calls that today go to a human rep or listening in as the customer describes her problem and fetching the information the human will need to solve the problem. The second scenario is especially good because it is measurable in two ways: not putting as many customers on hold saves money and increases customer satisfaction.

?

In the white-board phase it is important to be realistic about how accurate the AI needs to be to be a positive change and how much error the system can tolerate. In the doctor-patient example above, entries into the patient record need to be highly accurate which is why the AI only drafts entries and does not take any actions without the human saying to. In contrast, in the contact center if the AI can handle only a few more calls or only fetch the right information most of the time, it’s likely there will be little downside and significant gain. In my first post I discussed how AI gets better the more that it is used. The mechanisms differ between standard machine learning and generative AI but the upshot is the same: AI will get better the more that it is used so long as you’ve baked in a feedback loop. Launching an early version of an AI app is a great example of not letting the perfect be the enemy of the good. But that is also the reason than many organizations only make their V1 available internally and wait for improvement before putting it out into the world.

Keyboard: Only after getting very crisp on the problem and proposed solution does Noelle have anyone start coding. She’s adamant about avoiding scope creep, the tendency to try to solve many problems at once. In IT as in life, scope creep is a key driver of cost overruns and delay. ?Once her first project is a roaring success it’s easy to get resources for additional projects.

Noelle short hands the third stage as keyboarding, but as I’ve worked with organizations implementing generative AI I’ve learned that the keyboard phase is much more than coding. As organizations adopt GenAI they are often surprised at how involved front-line managers and other domain experts have to be. Typically these are the people who assemble the handbooks and other sources the GenAI needs. They write the test questions and answer them so the builders can test how accurate the system is. They also coach reluctant users to give new systems a try.

Hopefully as we reach the peak of the hype cycle more people will come to understand that generative AI is most useful when it s added to existing computer systems.

?