Evolution of Chatbots Leading to G-AI

In this post, I will review the evolution of chatbots and automation in sales and service areas.? Understanding where we came from is key to determining where we should go, especially in the euphoria stage with Generative AI (G-AI) in 2024. As Churchill said, “The farther backward we can look, the farther forward we can see”.

I will outline the more innovative moments in customer service automation and how that led to chatbots being the new interface for sales, service, and support use cases.? I will cover automation in Call Center Interactive Voice Response (IVRs), Webchat, Email, Messenger, WhatsApp, Slack, and of course the explosion of G-AI and how it's affecting these foundations.

Automation in Phone IVRs (Interactive Voice Response):

Call Centers were created as a knock-on effect of every household installing a phone line, and when there was a problem with a product or service, the household or business made a phone call to that company.? As call center volume increased, businesses started to offer automation tools to make their human agents more efficient, and the IVR was born. This IVR infographic highlights some of the technological breakthroughs over the last 90 years, starting from its inception in 1930.? There were a couple more specific technological innovations that helped pave the way for automation in call centers:?

1). Bell’s invention of Dual Tone Multi-Frequency (DTMF) in 1962 allowed phones to move from a rotary dial to a touch button dial. This innovation, along with a few more, allowed call centers to offer “press 1 for Sales, or press 2 for Service,” etc.?

2). In the 1980s IVR’s added basic speech recognition, which gave customers the choice of pressing buttons or speaking their responses. This represents the first audio type of chatbot automating tasks within the call center context.? While the first speech recognition system was not even close to where we are today in capability and performance, it is notable that call centers started down the road of chatbots in 1962 and the IVR market was sized at $4.5 Billion market in 2023 and continues to grow.?

Learnings of Automation in IVR:

What went well? Innovations were slow but steady, and over decades automation made a huge impact in scaling Sales & Service use cases for both Businesses and Consumers alike.

What didn't go so well? Too many companies have highly customized legacy IVR installations that are much harder to integrate into newer systems and tech.? Many IVRs are not contributing to, or benefiting from their companies' AI. This is mostly true for legacy businesses and not true for newer businesses with newer technologies.

IVR opportunity: ?Large Language Models (LLMs) have had many advancements over the last year, but the opportunity for IVR is text-to-audio and audio-to-text modality as multi-modality LLMs could increase the accuracy and effectiveness of IVRs generally.?

Automation in Email & Webchat:

In the 2000s, as Telecoms started offering broadband connectivity to consumers, businesses started producing websites, and internet access created more economy by driving more customers to those business websites. While call centers continued to receive increased phone call volume, net new audiences were also asking for support on websites via Webchat and Email. Companies like www.Zendesk.com were born to handle service and support via the Email channel, and companies like www.livesperson.com & www.intercom.com/ were born to handle Webchat. This was a pivotal moment in the history of automation as both Email and Webchat channels (derivatives of the Internet), were mostly text-based, allowing for more automation than was previously possible. Speech recognition in the early 2000’s was very limited, and touchpad dialing was also very rigid, but with more text input via Email and Webchat, automation tools had more context to process.

Webchat is a chat interface sitting on websites using web programming languages such as HTML, JavaScript with automated responses presented in button-like menus, making it easier for users to interact. While the Email channel provided more text input relative to Phone IVR, Webchat allowed for faster back-and-forth communication, increasing qualitative context while rich button User Interfaces(UI) increased interaction rates.

The early internet had dramatic impacts on Call Centers such as:

• Users moved from offline to the internet to buy goods and services
• Users graduated from only desktop to desktop and mobile devices, which in turn kicked off the outbound SMS industry with its own rules-based automation
• Call Centers added more software to address these new internet-based channels, and omnichannel SAAS was born

As call centers added more SAAS software, and as more software was moving from on-premise to the cloud, more automation became possible, and it also started to be applied across multiple channels versus just one channel. The automation in IVR remained rules-based and, in many cases, stayed on-premise, but Email and Webchat automation created additional opportunities to automate more tasks in deeper ways.

As human agents were routed to specific tickets, those issues generally had a predictable set of responses that could be placed in front of those human agents inside the SAAS software. Call center SAAS created a pivotal moment for automation, as it gave a new interface to the Human Agent, not just the end customer. This is when a distinction was made between “Automation for Users” versus “Automation for Agents". Another term for this today is “Human Assist AI”.

Learnings of Automation in Email and Webchat:

What went well? Text-based input allowed for more automation, and SAAS and cloud-based systems allowed for more flexibility and extensibility.

What didn't go so well? Email as a channel while still mainstream suffers from low open rates and conversion rates etc. Webchat times out(close) and is mainly adopted in desktop-heavy countries. Webchat allows for service and support only and doesn't allow for outbound use cases(such as Marketing).

Email & Webchat opportunity: ?Chatbots and Automation have been prevalent in these channels and with the advancements in LLMs more automation and efficiency are coming fast.? Natural Language Understanding (NLU), processing (NLP), and response (NLR) are now more cost-effective and attainable for all channels. This will result in bots being capable of human-like conversations, which will be more visible with Webchat and more behind the scenes with Email.?

Automation in Messenger, WhatsApp & WeChat:

In 2016, WeChat had 800 million users in China, and Messenger had approximately 900 million users globally. The below chart from Grate on X (formerly Twitter) posted in 2016, does a great job of showing all the features WeChat had before anyone else

WeChat was well ahead of the global tech titans, but WeChat was mostly operating in a closed market of China, so it was a phenomenon not well understood outside of China.? Both WhatsApp and Wechat rode the wave of free friend-to-friend messaging but WeChat added a Business API in 2013 which was the catalyst for major innovation and traction for B2C communications.? WeChat Business APIs were three years ahead of Messenger and five years ahead of WhatsApp

Facebook’s Messenger grabbed onto the automation capability seen in WeChat in China and Webchat in N.America, and during Facebook’s annual F8 conference in 2016 Messenger announced a few notable products:

1). Send/Receive API for businesses

2). Bot capability for Business (on the API above)

3). Wit.ai ’s bot for messenger consumers

A ChatBot hype cycle was born globally, and for the next few years, messenger chatbots were a dominant part of every Facebook F8 presentation and, subsequently, tech media publications.

In 2018 WhatsApp launched its Business API similar to Messenger but with the added benefit/complexity of being encrypted. Initially, the API was for non-promotional use cases with an emphasis on utility messages such as appointment reminders or delivery updates etc, while user-initiated messages were free for the first couple of years.

From 2016 to 2018 Facebook Messenger business-initiated messages were free, therefore Businesses messaged users liberally in the early years.? In 2020, Messenger went from being too loose to having more restrictions, and WhatsApp went from being too restrictive to loosening up a little. The revenue models remained different, WhatsApp mimicked SMS pricing models but was more expensive, while Messenger moved to more of a sponsored message model, which is more advertising-based, and not as targeted as an SMS model. WhatsApp has hit some Product Market Fit as business needs are more in line with the SMS structure.? WhatsApp spam rules are based on user negative feedback, which seems more scalable for consumers and businesses.

Business-initiated messages also result in some chatbot responses to help qualify respondents and nudge consumers toward a conversion. Messenger providers such as www.manychat.com and https://pancake.ph/ were early in deploying chatbots for Sales/Marketing use cases. ManyChat was one of the first providers in 2017 to offer a chatbot “flow builder” to its self-serve businesses, which was an important innovation in the area of chatbot automation, and most WhatsApp providers like www.Infobip.com and www.Gupshup.com also created a bot flow builder, albeit a few years later as WA API only launched in 2018.??

Facebook Messenger providers and WhatsApp providers typically deployed in different countries on different channels but invariably built the same messaging tech stacks such as:

1. Omnichannel APIs so businesses didn't have to rely on a single messaging channel
2. Chatbot “Flow builders”
3. Integration hubs for Enterprises…. or specializing in key tech stacks for Small and Medium Businesses (SMBs) like www.Shopify.com or www.Hubspot.com
4. Human-Agent Software
5. Analytics & Reporting

Automation was mostly deployed on components 2 & 4, although some automation was used in the other components to a lesser extent.

Other Messaging Options for Businesses:

GoogleRCS is now promoting its API to businesses in Android-heavy countries, which are also WhatsApp-heavy countries. RCS is following in WhatsApp API footsteps, which makes sense as WhatsApp has validated some Product Market Fit. Telcos worldwide tend to have SMS tech stacks that are only one-way, and Google RCS is offering telcos more 2-way messaging capability to help compete with the consumer messaging apps generally. Apple Business Chat is not as aggressive in its API GoToMarket but remains a critical solution in iOS-heavy markets like N.America, France, UK, Australia, etc.?

Learnings of Automation in WA, Messenger, WeChat:

What went well? Consumers really started to lean into their mobile devices; they weren't tied to landlines or desktops, and a conversation was always open. The advancement from users inputting text to pressing a button increased interaction rates. These benefits mostly occurred when one consumer messaging app was dominant, and businesses had an easy decision to lean in.

What didn't go so well? Business and therefore automation didn't hit Product Market Fit in countries where consumer messaging apps were fragmented such as N.America, France, and the UK. While automation improved in Email and Webchat channels, higher channel fragmentation reduced overall adoption and impact.

WA, Messenger opportunity: ?Chatbot tech stacks have been in place since 2014 (WeChat) 2016 (Messenger) 2018 (WhatsApp) with Product Market Fit in any country with consumer messaging app dominance. LLMs should be a huge accelerant for any messaging vendor already in-market with a decent solution, more details in part two of this post. Meta is also adding Generative-AI tools for their messaging users too, which will most likely scale in messaging groups versus 1:1 chats.

Business to Employee (B2E) Messaging

For brevity reasons I will use Slack as one example of B2E messaging, please note B2B messaging is much larger than Slack and Teams, and could be a separate post in the future.??

In 2012 Slack was born, and in 2021 Salesforce.com acquired Slack for $27 Billion, that's not a bad outcome in less than a decade and points to the high growth rates in the B2E space.? Anyone who has worked for a mid-size or enterprise business knows the importance of internal communications, and with the spam issues & long-form capabilities of Email, businesses globally needed a faster way to communicate internally. This blog post from Mio does an excellent job of documenting the history of Slack . But April 2019 was a pivotal automation moment for Slack when they launched the no-code app builder. This allowed for much more automation by any Slack Admin without manual code, in essence, Slack launched its own botflow builder, at about the same time that the WhatsApp B2C providers did the same on WhatsApp/Fb Messenger (ManyChat, Gupshup, Infobip).? B2E and B2C communication platforms applied chatbots at approximately the same time because the machine learning industry in conjunction with Web Hosting providers had reached a heightened point of capability.?

Today, Salesforce Einstein is being applied to Slack, and Open.AI and Azure AI are being applied to Teams, so I think it interesting to note that chatbots had been around the B2C & B2E ecosystem for years prior to Generative-AI (G-AI).

G-AI Enters the Timeline:

In November 2022, Open.ai changed the landscape by launching ChatGPT, as consumers globally could interact with a knowledge base of the internet and ask questions. Unlike Google or Bing Search, ChatGPT provides one human-like answer, versus a page of links, and a step change in AI research occurred as 100 million consumers flocked to ChatGPT in record time.? Consumers now have an expectation to ask questions and receive human-like answers.?

In the last decade, chatbots have received important but minor improvements, now with more powerful tech behind the scenes, there is a massive increase in chatbot capabilities, and everyone has taken notice across all industries.

G-AI as an Accelerant for communications:?

G-AI as an Accelerant to B2C Communications:

Some businesses have already transformed their communications by allowing their customers to decide which channels they want to communicate over, this is very apparent in countries where a messaging app is dominant such as WeChat in China or WhatsApp in Brazil. In mainland Europe, consumers tend to prefer to phone into call centers, and Businesses are slower to innovate in countries with regulations like GDPR.? Of course, Data privacy and security are paramount regardless of geographic region, but APAC and LATAM have leaned into Business Messaging more than mainland Europe has.? Mainland Europe and North America use Webchat and Call Centers as dominant channels and have not leaned into Messaging Apps, for different reasons.? Regardless, every business in every country is asking the same question, “how can I use G-AI and LLMs to serve my customers better and bring down costs”.? This is an accelerant to every Omni-channel Messaging provider regardless of channel or country because this specific industry has 5 to 7 very relevant years of experience, and G-AI is only one piece of the overall tech stack, meaning G-AI is not a silver bullet, at least not without the rest of the chatbot tech stack.

G-AI as an Accelerant to B2E Communications:

I see higher acceleration here as Azure AI is going to be tightly integrated into Teams, and Salesforce Einstein will do the same on Slack.? Businesses are more eager to enjoy internal productivity gains as there is an interpretation of fewer data security risks on internal communication systems versus external communication systems.? I see these two seemingly distinct sectors as two sides of the same coin and have more in common than meets the eye.

Why this Hype Cycle is Different:

Chatbots were hyped up in the Messenger ecosystem in 2018 and 2019, and one could argue they never lived up to that hype. Chatbots in WhatsApp & Slack didn't suffer from the same hype cycle. They invariably performed better because of the dominance of the WhatsApp consumer app internationally and the dire need for efficiency with internal communications.??

We are now in the middle of another chatbot hype cycle, and while the tech advancement is real, the providers who win or lose will be determined by how robust their tech stack is and their execution capability. Winners will be the ones who execute a robust solution, not just market it, which has always been true but is more acute at the tail end of hype cycles.

The reasons why this chatbot hype cycle is different are:?

• The automation capabilities of the bots are now far superior than they were in 2016. In fact, progress in automation capabilities in 2023 far exceeded capabilities in 2022, signaling a step change from rules-based to generative dialogs.?
• There is already a chatbot industry in place, whereas in 2016 it was nascent. This means the whole ecosystem is more mature, not just in front-end UI but also in backend hosting, databases, Dev Ops, etc, and not just in vendor capability but also in enterprise customer sophistication, etc. The chatbot industry is far wider now.
• Human behavior and expectations have changed.? In 2016, mediocre bots with rigid flows mostly frustrated customers, but now, the customer really doesn't care if it's a human or bot, as long as they can resolve their issue or accomplish their task. Open.ai ’s chatbot has changed user expectations and there is no going back.

In part two of this series, I will dive into how business messaging providers can extract value from Generative AI and thrive in this new wave of technological progress. I will explore the debate “if older dialog flows are still useful or does G-AI forgo rigid flows completely and orchestrate on the fly…“? The next post will be a fun one, please ping me with any opinions of your own.

A big thank you to a wide set of industry experts who discussed or debated some of the above topics. These folks include: Thomas Kuruvilla , Alex Roucourt , Ido Bornstein-Hacohen , Veit Irtenkauf , Mike Yan , Brett Weigl , Ze'ev Rosenstein , Anand Thaker