What’s in a LLM? The public vs private debate

Most people by now have heard about #ChatGPT.?

The sprawling Large Language Model (LLM) that chats with you using human-like dialogue has sparked a ton of excitement (and maybe a bit of anxiety).?

As a public LLM, ChatGPT is trained on public data vacuumed up from all corners of the Internet. This gives it access to an enormous amount of information that generates a pretty comprehensive range of answers to almost every question you can throw at it.

What most people may not be aware of, though, is that there are also private LLMs.?

Unlike public LLMs, private LLMs are trained on a ring-fenced source of data that’s not available to the public. Typically, this will be data created by an organization with a fairly specific expertise.?

A good example of this is the private LLM recently created by Bloomberg (BloombergGPT) that’s trained on the company’s repository of finance-specific information. Or, let’s say you’re running a #contactcenter. Theoretically, you could have a private LLM that’s trained specifically on your business conversations.

So what’s the benefit of that, compared to using something like ChatGPT?

Well, for one, it’s true that public LLMs can draw on the vast resources of the Internet to answer your questions. As we all know from experience, though, the Internet can cough up some pretty weird and inaccurate answers.

A private LLM, by contrast, is trained on a very specific, focused and vetted set of industry-specific data such as finance or medicine. This “cleaner” data is going to give you much more reliable and relevant answers to questions in those fields.

Going back to our contact center example: Say you’re running a customer-facing contact center and want to use a #generativeAI tool that automatically suggests responses for your agents when they’re talking to customers. This AI tool would be powered by an LLM.?

Every organization has its own way of communicating with its customers and sales prospects involving unique products, terminology, and style. Because of this, a private #LLM that’s trained specifically within the bounds of that organization or industry will give you much better results than a general model that’s scouring the whole Internet and learning from data sources like YouTube or Reddit.?

“They can be laser-focused on the kinds of business conversations and applications that a company is really interested in,” says Simon Corston-Oliver, Dialpad’s Director of Machine Learning. The thing is, even though we’ve all played with ChatGPT and gotten it to generate some pretty interesting responses, this doesn’t necessarily translate into a business use case.?

“Companies are typically not interested in things like writing a haiku about a meeting, a story in Shakespearean English, or the other gimmicky things you see elsewhere,” says Simon.

Here’s something else to consider. Public LLMs are bit of a two-way street. While they have access to an ocean of Internet data, your interactions with a public LLM could be causing your own information to trickle into that ocean where it can then be used by others. That’s why a number of large companies have banned their employees from using these tools.

“This commingling of data is a worry,” says Simon, “If you can keep your data private and in-house, then you have peace of mind when it comes to dealing with the privacy concerns of both your customers and regulators.”

This is really just a small glimpse of the possibilities that are emerging with business-specific, private LLMs. And that’s not even considering the rate limits for businesses that want to use ChatGPT, the associated compute costs, and the huge environmental costs of using public LLMs compared to smaller, more efficient private LLMs.?

Has your company started experimenting with LLMs —and if given the choice, would you be likely to use an industry-specific private LLM instead?