From Chatbots to ChatGPT ?? - my story building conversational features and products

want to take you on an a journey into the world of conversational AI, a world I have been diving in and out for the past years. Specifically, I will share with you my experience at Accor, where I was leading the Product Innovation Factory team, building a robust conversational platform for the Accor group, and how everything has changed 4 years later with the arrival of chatGPT with 2 real use cases at Cityscoot of chatGPT (yes, real ones!).

I. Chatbots and chatGPT

Chatbots in 2019

Chatbots have been a game-changer in many areas like customer service, providing automated responses and support to users. (Ok, ok, some of them are very poorly developped and drive us mad... I know... ??).

During my time at Accor, building the conversational platform was a monumental task. We had to construct a comprehensive architecture from scratch, incorporating various technologies such as Rasa, DialogFlow, and AWS. Using new messaging channels like WhatsApp, Messenger, and Quicktext to ensure seamless communication across platforms with our users. It was a complex puzzle that required constant attention to detail.

A traditional conversational platform would include the following components:

• Speech-to-text and Text-to-speech engines: if we use voice/sound, these blocks will convert to/from into text.
• User interface: usually messaging platforms like Whatsapp, Messenger, or a chat in a website
• Backend for textual input & output: receive and send text responses from UI to the language engines.
• NLU (Natural Language Understanding) or NLP (Natural Language Processing/Programming) : will interpret the text and extract meaning --> Requires high effort to train and adapt
• Decision engine: will decide what to do with the understood information --> moderate effort to program
• NLG (Natural Language Generation) : will generate a human-like response --> high effort to train and adapt

To keep this system up to date, I had up to 2 NLU/NLP engineers in my team (amazing guys by the way who shifted paths afterwards and grew up to become awesome Product Managers). They spent their days fine-tuning the models, training, and recreating decision trees to enhance the platform's capabilities....

Enter chatGPT

Note: I am not going to talk about the ins-and-outs of chatGPT, LLMs and the technology underneath, maybe in another article... I will solely focus on the business / functional aspects.

Unlike traditional chatbot development, ChatGPT simplifies the process by offering a plug-and-play solution already trained with general knowledge replacing the Natural Language Engine block in the previous drawing and its extensive need for training. With its vast language understanding capabilities, it can respond to questions even if they haven't been explicitly trained by the user (as long as its general knowledge covers it) and it is capable of categorisation even outside of the provided examples. This can accelerate development of solutions without constant manual training or tweaking.

Beware of the big bad wolf...

While the potential of ChatGPT is immense, it's essential to navigate its opportunities and risks responsibly. A lot of question around ethics and explicability arise when using chatGPT :

• Regularly assess the veracity of responses, as chatGPT might answer false, incomplete or unverified facts. It can even promise things that are not feasible (I will dive into some examples later ??).
• Explicit the fact that it is not a human replying: responses might be so human-like (unlike old chatbots) that a users might not make the difference, hence the important of stating it.
• Weight on the non explicability: you will be never know exactly what the answer will be despite setting up conditions and boundaries to the prompt. Is the loss of complete control acceptable for you?
• Beware of bias amplification: as you do not control the baseline training or the provided answers, biases (unfairness, discrimination, etc.) can occur without your explicit knowledge.
• Protect your data: chatGPT is not known for protecting personal or confidential data, and there have already been cases of security breaches at OpenAI... Make sure the information you feed chatGPT is safe to be made "public" (non confidential, anonymized, etc.).

II. Real world examples using chatGPT

We have been exploring chatGPTs capacities lately at Cityscoot and leveraging Proof of Concepts using openAI's APIs and low-code solutions like Node-red. My PMs have some basic knowledge of programming and are not scared of diving deep with low-code tools, so we launched several Hackathons ("build a product in half a day") to experiment with chatGPT.

Note: I am a big fan of Node-red and introduced it to my teams. Take a look at an article from Michael in team where he explains how using a basic React front-end website and Node-red as backend he could create an innovative product to declare badly parked mopeds in the city of Paris, in record time.

Use case 1: Analyzing, categorizing, and responding to end-of-ride customer feedback (= a problem of volume)

Have you ever used Cityscoot's app? Last year we launched an end-of-ride feedback feature (that I truly pushed for as it gives constant quantitative data of user satisfaction and many other metrics to take our decisions and shape our strategy).

Almost half of our users send us a feedback (that's great!). With roughly 10 000 rides per day in Paris, that's 5 000 feedbacks per day... Most of them are just the Like/Unlike button (smiley faces in our case), but some users take the time to declare the source of the problem and even write a comment (saying for example that the moped was in bad shape, there was a problem in the billing amount or any other problem).

As you can image there were too many to be read by humans, so up to now we only did basic statistical analysis on the results. Our users were not happy, thinking that their comments were dealt with and realizing that no action was taken.

Using node-red and chatGPT's API, we built a process in two steps for customers sending an unhappy smiley feedback with a written comment:

1. give chatGPT a list of possible categories and incidents with the corresponding codes that FreshDesk (the ticketing system used by our customer care) can understand, and let it extract the list of the matching ones in the user's comment.
2. provide a human-like empathetic answer generated by chatGPT to the customer

Of course we had to set up some rules/boundaries for the answer:

• Explicitly sign as "Cityscoot's virtual assistant" so that the user doesn't think that a human responded
• Make sure the customer understands that this message does not mean that a human is actually performing an action and if he/she requires or expects an action, then provide the phone number and email address to contact customer care
• We never send any personal information about the customer or the ride itself to chatGPT.

Note: After some trials & errors, we noticed chatGPT was way too empathetic and promising impossible things, like "don't worry, we will send an agent in 5 minutes", or "be reassured we will call you back in the minutes to solve this"... we had to add more rules like "do not promise or engage in any action on our side, if the user requires an action, he can contact customer care".

Prompt engineering is an art ! You cannot get it right the first time and with trial and error, you can refine your conditions to make sure your generative AI does not go out-of-boundaries.

As of today, we have a 66% of positive client feedback to our chatGPT responses, not bad at all! ??

But of course some questions arise:

• Are we sure chatGPT is properly categorizing the elements and properly interpreting the comment?
• Are we sure that it is not providing impossible solutions or incorrect answers to our customers?

The answer to both questions is: it cannot be worse than before (we weren't doing anything) ?? luckily setting up the good boundaries should prevent a complete freestyling spree of chatGPT ?? (here is the "leap of faith" part of using this kind of technology I suppose ??)

Use case 2: Automating analysis of operator declarations in the field (= a problem of time-to-market)

People see our scooters, our app, but what they don't see is everything behind the curtains... More than 70% of our staff are workshop workers, battery swappers, field agents, etc...

Now picture a field agent, trying to diagnose a broken scooter in January, -2oC outside, under the rain, in a busy street in Paris... He has to take out his smartphone and fill up an incident form with the diagnosis, with his bare freezing fingers on a wet screen... Wouldn't it be much easier to use a vocal command to declare the incident and the form filled automatically?

This has been in the pipes since I arrived, but I knew how much work this kind of feature would require with the tools I knew from Accor... But the game changed with chatGPT! ??

I leveraged my knowledge of Ionic Capacitor (Angular) to build a quick prototype app that would use Android's native Speech-to-Text function, letting us focus on the backends' node-red/chatGPT logic.

One node-red backend endpoint would take the provided transcribed text and ask chatGPT to extract the corresponding items that are declared and their criticity. Of course a list of possible items and criticity criteria are previously provided to chatGPT. Anything that does not fit a category/item will go to "Other" and be summarized in a String field.

The user then receives the list of items than he/she can modify (it is important that this information is accurate). When all is OK, then a second node-red endpoint receives this list, fill the form data in the database and actions can be automatically trigered depending on criticity levels.

Note that the process of transcribing and categorizing is human-validated, since the operator validates the proposed items from the list, removing the question of trust from chatGPT.

In one afternoon we built a solution that would have taken my former team months to train! ??

If you enjoyed this article, I'd appreciate it if you hit the "Like" button and share it with your network. Together, we can inspire and empower others in their journey through the evolving landscape of AI. ??