5 Questions to Consider Before Investing in LLM AI-powered Chatbots for Customer Experience

Over half a decade ago, it was my job to think (and talk) about how to use conversational AI to improve the customer experience. Chatbots were all the hype, although most commercially available solutions did not have very advanced LLM-based engines at that time. For some time, it became quieter, but with the rise of ChatGPT and other "large language model" (LLM) applications, it's the talk of the town again.

Like many, I'm impressed with the quality of the conversations. Maybe even more so, knowing it is not a machine following a predefined path of "if this, then that" strings in a structured knowledge base application, but the power of machines learning unsupervised how to interpret questions and answer them.

This begs the question: what does this mean for customer experience design? Will LLM-powered chatbots become the new interface for organizations when customers interact with them?

I don't have a definite answer to this question because it depends on many factors. But based on my experience, reading, and recent conversations with experts like Brian Manusama (and asking ChatGPT, of course), I can give you 5 questions to help you decide whether it makes sense or not for you to invest time and money into this right now.

Q1: Do you have time and money?

Organizations have two objectives when considering using conversational AI: improving the customer experience and reducing the costs to serve their customers. I believe it is fair to expect that LLM-powered chatbots will improve the customer experience. If you have experienced conversations with ChatGPT yourself, you know this. It is just way better than any chatbot experience you have had before. Yet, the costs of achieving this level of experience might be very high and take a lot of time.

LLMs need a lot of data, computing power, training time, and people capable of setting all this up. All of that comes at a cost. SaaS solutions will likely be around, but there are costs and a significant timeline for (implementing) that too. 6 to 9 months is probably an optimistic timeline. This all increases if you want to integrate data from your CRM/ERP systems in the conversations to further personalize the customer experience and enable the chatbot to execute even the simplest of tasks.

In other words: If you need benefits fast and/or money is tight, this is not your project. If you have time and a sizable budget, you could try to get things moving, provided you have a viable use case.

Q2: Do you have a viable use case?

LLMs need a lot of data to be trained properly, preferably from actual conversations like live chats, e-mail conversations, online Q&A forums, etc. LLMs also like it when the data covers a wide range of topics. If you have millions of these conversations stored somewhere, you may have a no-brainer from this perspective. If you have way less, let's say just a hundred thousand, or two maybe, it may be tricky. It becomes specifically tricky, if not impossible if you do not have vast amounts of data AND the data covers many different topics. Simply put, if you do not have millions of customers/users that interact with you regularly, this may become a difficult case.

On the other side, if you can find a use case where you have lots of data that covers a few topics, you may have a winner. FAQs might be a good place to start. But before you get carried away, I can advise you to check if your current way of dealing with those FAQs is already good enough to satisfy your customers and prevent them from resorting to live contact. Because if that’s the case, your business case may not be positive as well.

Another opportunity area may be in the early stages of the customer lifecycle when customers are evaluating their options to buy a product or service and are trying to get a grip on their decision. In this stage, having a conversation with a “synthetic” agent advisor might be just what they need when live agents are not available or too expensive. And the business case is always easier when growth is involved.

Q3: Do you change things a lot?

However, in the current marketplace, companies and governments change products, services, conditions, and procedures all the time to adapt to changing customer needs, etc. LLMs will not automatically update their response to customer questions when you ‘air’ a product update.

The LLM requires (re-)training from a fair amount of high-quality examples. This probably means you need to integrate other solutions into your chatbot if you want it to be a trustworthy conversational agent advisor for your customers. And you should have live-agent escalation possibilities available. These live chats could then serve as training material for your LLM, provided your product or process doesn't change before the LLM is done learning, of course.

Q4: Is your customer ready?

LLMs appear to be great at providing answers to precise questions, or prompts as they are called in ChatGPT. Many of the tips and tricks going around on the internet are about these prompts and how to write them in such a way that you get the answers you need. But if you look at the search queries that people use on your website or even with your current chatbot, I think it is safe to say that customers need to get used to this new way of interacting with your LLM-powered bot.

Many customers use keywords only or just short sentences that are very generic instead of specific. This will make it difficult for the LLM to present accurate answers that are true for the specific situation of the customer, which is what most customers want.

And of course, when more training data with a wide variety of customer inputs, linked to the right outputs, is available, the better the LLM will be able to deal with ill-structured sentences, vague descriptions, etc. Nevertheless, you should be aware that you may need to help your customers structure and detail their questions a bit to get the best answers and experience. The unknown here is how customers respond to that. Do they value the experience and the output, or not? And what can you do when the answer to that question is “not so much”?

Q5: Is your company ready?

Besides all the practical challenges, there are also risk-oriented, cultural, ethical, and ecological challenges to take into account. Your organization’s PR, legal, compliance, and sustainability teams may have strong opinions, and I would not disregard public opinion on this matter as well.

At the most fundamental level, there is a fear that AI will become so powerful that it will run itself, and humans will have little control over it. Is your company ready to cede control of the ability to learn and provide answers to any type of question? Are you willing to take legal responsibility for the consequences of decisions your customers make based on an LLM's recommendation?

Furthermore, how can you ensure that the LLM is not biased in its answers and recommendations to certain groups in your user/customer base? For instance, what if men have always been your main interacting customers, but you want to target women more?

A lot is unknown, and if you are to start experimenting, you should consider this perspective and conduct some fact-finding.

Last, but certainly not least, doubling down on AI will hurt your carbon footprint because of the vast amounts of data that need to be processed. This needs to be addressed or the board’s public promise to the markets that they will achieve net zero by X date may be at risk.

To conclude:

I am confident that LLM-powered chatbots have a bright future, and there will likely be successful use cases in the field of CX and customer service shortly. However, this may not be true for every organization. Whether this is an opportunity for your organization now depends on your answers to the five questions above.

If you are currently exploring this topic or feel intrigued and want to make this assessment for your organization, feel free to contact me. I am happy to help