Benefits of a Chatbot Pilot Program

Contents 

Abstract

Part 1: What Chatbots Can Do

Part 2: Four Types of Chatbots

Part 3: Chatbot Pilot Use Case

Conclusion

Abstract: Meet the new customer service agents

Chatbots leverage Artificial Intelligence to simulate and process human interactions.

Can you imagine a situation where all interactions between citizens and government agencies take place online? This is the reality many agencies faced during the pandemic. The public sector had to adapt almost instantly to provide citizens with virtual access to services.

As a result of this sudden shift, departments needed to provide consistency in internal operations, as well as meet the demand for better online customer service. One popular solution has been the adoption of chatbots or virtual assistants.

These tools are continuing to be embraced in the post-pandemic world. According to Business Insider, the global chatbot market is anticipated to grow nearly 30% by 2024.

So what exactly is a chatbot? In short, it's a tool that often leverages Artificial Intelligence (AI) to simulate and process human interactions. Virtual assistants, such Apple's Siri or Amazon's Alexa, and customer service chatbots can be programmed to converse with users via text and voice, allowing citizens to more easily interact with a department's information or services.

A chatbot pilot helps to define the best solution, and refine the features needed to be most effective.

While these virtual communicators are not a perfect substitute for human conversation or support, they are rapidly becoming more human- like, and better at responding 24/7 to citizen needs in nuanced ways.

As it turns out, not all chatbots are created equal. Each is designed for a different purpose, with different levels of depth and complexity. When considering adding a chatbot to a website, the first question clients ask is: How do we know which type of chatbot is right for our needs?

Development of an effective chatbot can range widely in time and cost, so to prevent overspending, it's vital to determine what style and features will best match the situation.

As we have found, the planning and implementation of a chatbot pilot helps to define the ideal solution, and refine the features needed to be most effective for the core audience(s).

Part 1: What Chatbots Can Do

Connecting users to information with less friction.

First, let's take a step back and look at what chatbots can do. On public sector agency websites, most chatbots use AI and machine learning technology to process large volumes of data and respond appropriately. Functioning as a robust search tool, they enable users to search for and connect with the information they seek with less friction than standard navigation or search. Integrated as a pop-up on a department's website in addition to the traditional search bar, they make it easier to navigate large amounts of content. Chatbots can also boost operational efficiency when used internally to aid with staff and departmental activities.

According to Review42:

50% of all searches were voice activated in 2020

55% of households in the US are expected to own a smart speaker by 2022

20% of mobile queries are currently done with voice search

Prescribed rules, scripts, and machine learning can create responses that are both highly consistent, and develop over time.

Additionally, chatbots are available 24/7 to multiple users simultaneously, and can be programmed to be either proactive, or reactive in their interactions. Prescribed rules, scripts, and machine learning can create responses that are both highly consistent, and develop over time. For instance, using the initial information loaded into the program for question and answer pairings (intents and responses), a chatbot with machine learning can then build upon these basic responses, so that the service provided becomes more valuable over time. The program learns from experience so it can provide a wider range of responses to similar questions, or can understand more variables of the same question. As a result, natural language processing delivers responses that are more ‘human’ in nature and less scripted.

GPT3 (Generative Pre-trained Transformer) is one of the more exciting technologies used to provide human- like responses. Developed by OpenAI and launched in July 2020, GPT3 utilizes over 175 billion machine learning parameters (about 10 times more than Microsoft’s Turing NLG) to learn from thousands to millions of examples in a "structure loosely based on the neural architecture of the brain."

Based on deep learning and natural language processing, it requires very little input or training to create human- like content and text. GPT3 uses full language understanding, or natural speech recognition, to recognize open ended questions.

Regardless of the technologies used, chatbots help citizens engage with the information they seek, and surface other relevant information, which may not have been considered originally. Furthermore, chatbots give staff time to provide more personal human interactions to the individuals who need to speak with a live person to resolve an issue.

90% of respondents noted enhanced call volume processing using virtual assistants. –MIT Technology Review 

Part 2: Four Types of Chatbots

Options span a range of costs and capabilities.

Now let's review the various types of chatbots to be explored during the pilot planning stages. The business use case will determine if the chatbot should respond to text, visual, or voice cues. Furthermore, it will guide the choice of chatbot type, from knowledge base and rule-based to conversational and hybrid models.

Knowledge base: A knowledge base (or FAQ) chatbot relies on question and answer pairings to provide suitable responses to a given topic. In this regard, it is similar to a robust FAQ database, which uses a library of questions and answers within a set framework or structure.

The limitation is that they can only process questions that have answer pairings, but since knowledge base chatbots meet a specific need, they are very popular on agency websites. Clients implement them for 24-hour support for audiences and to reduce support costs. Existing FAQ content can make for a quick turnaround, with several chatbots deployed on one site for different divisions or sections of interest.

Rule-based: Rule-based chatbots are slightly more detailed in that they offer users a choice with buttons or verbal options. Essentially, they are complex decision trees with rules triggered by specific words or phrases. For example, a prompt such as, “How do I apply for...” may result in a variety of relevant answer pairings. On some, the button will match a defined intent. On others there will be a direct link between the button and the response. When a user types a question or clicks on a button, the user interface either provides the most probable response, asks for further clarification, or delivers a range of possible answers. This type of chatbot is most useful to help a user navigate through multiple layers of content.

Not only can staff time be reallocated with hybrid chatbots, but the department can refine their communications with the data collected.

Conversational: AI technologies such as machine learning and natural language processing allow conversational chatbots to search deeper knowledge bases to provide the most appropriate responses. Additionally, they can remember conversations with specific users, and add this data to the knowledge base for future use. Predictive intelligence can also surface recommendations based on the user's previous interactions, similar to the AI used by content providers like Hulu, Netflix, or Siri. Drawing from many disparate systems, conversational chatbots provide a streamlined experience for the user, presenting only the most relevant content based on the context of the interaction and the question being asked.

Hybrid: The hybrid model contains some rule-based and scripted features, as well as natural language processing and machine learning to accumulate a broader dataset for subsequent users. Because these data-driven and AI- enabled chatbots often use predictive features to provide more conversational responses, these bots can understand the context of the questions being asked and the user intent, which they learn over time. Like the conversational bot, they are able to respond with complete answers, which become more useful as they learn more information about the user. As a result of the more tailored customer service experience, not only can staff time be reallocated, but the department can refine their communications with data collected.

$4.4 billion spent on chatbots in 2020. –Markets and Markets 

Part 3: Chatbot Pilot Use Case

A pilot approach helps to manage scope and costs.

The following use case looks at how a chatbot pilot can sample the technologies available in order to provide the most relevant minimum viable product (MVP) to serve a specific audience. "A well crafted pilot program is a low-risk way to roll out chatbot and AI assistant solutions to your organization," says GovWebworks chatbot developer Adam Kempler.

"It allows an agency to put a toe in the water and trial the technology in a specific situation to see if it might work in other areas as well. Clients appreciate the pilot approach because it helps to manage scope and costs with a smaller subset of requirements.

Furthermore, you can find an area that will meet the need of a particular audience with a specific functionality and capabilities, and build a pilot around it to get feedback from the audience as well as internally to see if the approach is effective."

For example, the Missouri Department of Conservation was looking to explore the use of chatbots to help users find information on their site and reduce repetitive support calls. A hybrid chatbot was prototyped allowing users to find information on hunting, season dates, methods, and other species-related information. Additionally, Webworks is helping the Minnesota Department of Employment and Economic Development (DEED) to implement a chatbot pilot so people who have been laid off can find out if they are eligible for additional benefits.

Identifying how to reach your target audience is the primary factor in deciding what channel will be used.

When planning a chatbot, there are two primary channels to consider, text input and voice. These are not exclusive, and should be selected based on how the audience is looking to access the information. Identifying how to best reach your target audience is the primary factor in deciding what channel will be used for your pilot. It can be easiest to try a single channel first and add another channel later. Text input is a common starting point, as many audiences are already interacting with websites in this manner.

For agencies who want to explore the options and implement a chatbot pilot program, chatbot developer Adam Kempler recommends the stages and steps outlined on the following pages.

Talking with customer support staff to list and categorize major communication gaps and common questions is a good place to start.

Step 1: Planning

1. Identify a key pain point: Sometimes the need for a chatbot is obvious, other times, there are competing pain points. Identifying the top pain point helps scope the solution. Talking with customer support staff to list and categorize major communication gaps and common questions is a good place to start. Try to find a set of consistently asked questions which could be answered in a consistent way. Ideally, these questions and answers would be related, and have a consistent audience. This will allow your pilot to be more focused and to control scope.

2. Create a project brief: This helps define your primary objectives, and audience(s) for the chatbot. For example, an objective might be to reduce the number of calls to customer support for questions like, “how do I apply for...”. Limiting the number of objectives and making them measurable will help you track the effectiveness of the pilot. The key is to keep the scope small and focused - you can always expand later.

3. Determine the appropriate channel: This could be voice or text input. Voice platform examples include Alexa and Google Home, while text input examples include platforms such as Facebook Messenger, Slack, and Telegram. A chatbot embedded on your website would also be an example of text input.

4. Choose the appropriate platform: The goal of the pilot is to successfully reach the primary audience. Start with one platform at a time. This could be via an embedded chatbot on your website, or Facebook Messenger. You can grow the chatbot on other channels later.

Think about the kinds of data your chatbot will need to access, where that data currently lives, and how often it is updated.

Step 2: Architecting

1. Design the conversation: A simple spreadsheet can help to define the steps that a user would take to get to the information they are seeking, what questions they might ask. This will help identify the best follow-up questions and replies for your chatbot.

2. Identify triggers: Chatbots can be triggered to engage with users automatically (such as when a user spends more than 20 seconds on pages tagged with a particular keyword or based on certain search terms), or manually with some form of visual icon/graphic that a user can click on to initiate the chatbot window.

3. Identify hand-off options: If the chatbot is unable to help the user, it will need a way connect with an actual person.

4. Determine backend integration requirements: Once the conversation flow has been defined, you will also need to identify any backend integration requirements. Think about the kinds of data that your chatbot will need to access, where that data currently lives, and how often it is updated.

5. Locate the best tools for the job: The number of tools and platforms for implementing chatbots is seeing explosive growth, from open source libraries to enterprise ready systems. Criteria that can influence your choice include; price, software code(s) supported, backend integration capabilities, and ease of management.

Your prototype should be functional enough to assess whether the requirements identified during planning stages meet the criteria.

Step 3: Develop and test

1. Develop a prototype: As stated before, it is best to start simple, gather feedback, and iterate. Your prototype should be a rough implementation of the final chatbot. Clarify with stakeholders early in the process that the prototype is a very rough, unfinished version of the pilot chatbot. It should be functional enough to assess whether the requirements identified during planning stages meet the following criteria:

• Refine the prototype and develop the fully functional chatbot to use for your pilot. Additionally, identify resources such as who is going to “train” the chatbot.
• Conversation design correctly captures the necessary dialogs Technology choices meet the identified needs
• Invite a variety of users test your chatbot. Verify that the chatbot is guiding users to the correct information. Document where and how the conversations break down so you can make modifications.
• Targeted channels and platforms are supported
• Integrated services, including internal and third party services, are supported
• Stakeholder needs are met

2. Invite stakeholder feedback: Review the prototype with internal stakeholders and get feedback as necessary. Weigh any new feature recommendations or change requests to manage scope. Changes at this stage cost less than after completion.

3. Develop the pilot chatbot: Refine the prototype and develop the fully functional chatbot to use for your pilot. Additionally, identify resources such as who is going to “train” the chatbot.

4. Test: Invite a variety of users test your chatbot. Verify that the chatbot is guiding users to the correct information. Document where and how the conversations break down so you can make modifications.

5. Assess, refine, expand: Most natural language processing platforms will provide some mechanism for reporting and training. This is where you can view what terms and phrases your users are typing or saying to the chatbot. Check to see that user questions were correctly matched to chatbot dialogs, or if the chatbot was unable to match to a valid response. You can then enhance and refine your chatbot conversation capabilities as necessary.

Conclusion: Chabots are here to stay

Agencies can begin with a pilot chatbot to demonstrate success and prove value.

It's inevitable that departments will add more AI tools as they continue to enhance and adapt their operations. In a post-Covid-19, digital-first environment, chatbot tools will become more pervasive and robust, with abilities that are increasingly similar to human interactions.

As agencies decide which bots best meet their needs, and where to deploy them, they will find enhanced benefits including faster customer response and less pressure on staff. This will in turn free up staff to handle work that needs a human touch.

In the years to come, we expect agencies will have several forms of AI technology in use. Departments will embrace machine learning and natural language processing to make the digital world feel more personal and real. Additionally, agencies will be able to offload more of their administrative activities and communications using digital assistants and AI connected technologies.

As detailed this white paper, agencies needn't be afraid of these AI tools, but can begin with a pilot chatbot to demonstrate success and prove value.

Author bio

Ravi Jackson has an extensive professional background in law, finance, policy, and the technology sectors.

This post originally appeared on the blog page of my former company, GovWebworks at "Benefits of a Chatbot Pilot Program"