Bots and Chat bots –Revolution in Software Testing

What is a bot and Chatter (chat) bots?

We are familiar with AI (Artificial Intelligence) concept these days. We also getting familiar with computer assistant popping up in our apps, websites while booking flights, car, banking, searching something specific etc... Those are Virtual assistants called Bots much like Alexa, Siri and Cortana, but they communicate via text rather than speech.

A bot (short for "robot") is a program that operates as an agent for a user or another program or simulates a human activity.

Bots aren’t new, one of the first and most famous chatterbots (prior to the Web) was Eliza, a program that pretended to be a psychotherapist and answered questions with other questions.

A chatbot is an artificial intelligence (AI) software that can simulate a conversation (or a chat) with a user in natural language through messaging applications, websites, and mobile apps or through the telephone.

Google Assistant, Alexa, Siri etc... Are AI based Chat bots.

Bots around us - It feels family!

Let’s have wonderful example of a bot i.e. Nissan's new Micra chatbot. It asks questions about your personality and tells you what colour car you should be driving. Sound’s interesting right! Designing your own car with the help of bots?

Bots let you use natural language to get tasks done. This is one of the reasons many people use Siri or Cortana to check the weather forecast, set a reminder or send an email: it’s just faster. I feel them as part of my family now.

Chatbots are growing in popularity with each passing day. The rise in chatbots over the past few years has been due to increasing competition.

Testing of Chat bots

Chatbots are growing in popularity with each passing day. The rise of chatbots has also come with a set of challenges for testers and developers.

To avoid any discrepancies and turning down the customer, chatbot testing must be a top priority for developers and of course testers.

Next-generation interfaces should be tested for behaviour, dialogue accuracy, speed, compatibility across platforms and collecting the right data.

Arbon, who also penned the book How Google Tests Software, says testing is harder than writing software. “You have to be smarter than the programmer to find problems in the code.”

Types of testing to include for bots testing:

• Usability
• Developer testing
• User testing
• Speech
• Behaviour
• Functional
• Precision
• Security
• Load
• Localization and Multi-lingual
• NLP capabilities

Some of the features that need to be tested are:

• Conversational Flow [Google Dialog flow Chatbot Testing]
• Understanding/Intent
• Error Management
• User-friendliness
• Bot Speed [Minimum Latency in response retrieval]
• Bot Accuracy
• Navigation -Hyperlinks in response direct to the correct site
• Avoid cyclic loop

Techniques for bots testing:

Pyramid Approach -3 broad steps:

General Test: the funnel for testing should follow an inverse pyramid shape. Question and answer testing would follow that structure. Chatbots are expected to keep the conversation flowing, if they fail at the first stage then, the user will likely to leave the conversation.

Domain Specific Test: testing for the specific product or service group. The language and expressions related to the product will be the main driving force for the test.

Limit Test: Testing the limits of our chatbot. Will our bot will able to handle the user’s irrelevant info and dialogues?

Chatbottest’s Standardized Test:

The test mechanism developed broadly follows three categories. Expected scenarios, possible scenarios, and almost impossible scenarios.

Human Testing of Chatbots

Developers can leverage this service to build human intelligence directly into their applications. This service can be used for further testing and reach for a higher confidence interval. Like using Amazon’s Mechanical Turk which builds higher confidence intervals and performance.

Now we have talked about how we test and what we should test, but should you implement a chatbot? Before answering the components of decision, let us first look at the benefits of having bots –

• It can work 24*7, is quicker and more efficient, and much less prone to error.
• Chatbot applications streamline interactions between people and services.
• Enhancing customer experience.
• Improve the customer’s engagement process and operational efficiency by reducing the typical cost of customer service - chatbots makes it easier for firms to reach customer’s expectations.
• Can solve many of customers’ problems with traditional online experiences.
• Make it possible to maintain the same service quality, no matter what the circumstances are.
• Provides suggestions, powered by greater analytical capabilities.
• Generates a greater revenue than the traditional methods.

Thinking to implement one for your organisation? Here are the key components before taking decision –

·        Decide on the value you are expected to gain by implementing a chatbot.

·        The medium the chatbot implemented is another factor. Will it be voice-based or text-based?

·        What are the available data to train chatbot?

 So Turn on your chatbot into success with above mentioned areas of testing (very important)

Few tools in market for chatbot testing:

• Testyourbot
• Botium [Selenium for chat bots]
• QBox.ai
• AI testing bots tools with Automated AI

Staying ahead of the bots

At the end of the day, the best tested chatbot is that one that makes users feel engaged and entertained and that leaves a positive impression about the brand.

In an age of constant evolution, it's no surprise that AI-driven solutions have come to help us with our jobs. How will we stay ahead? More importantly, how can we become even more effective by leveraging the power of upcoming machine-learning test tools? If you take a moment to think about all the technologies we use on a daily basis, AI has already begun silently integrating into our lives. Ready for revolution?