Are we sure we know what our customers want?

We need better ways of getting user feedback

There’s no shortage of articles talking about the benefits of AI to consumers. AI will provide hyper personalised services in shopping, music, films and more. AI will create self-driving cars no nobody will have to learn to drive. AI will automatically manage the temperature in your home. But do consumers agree? I’ve written before about the fears certain people have about AI but there’s another perspective on this. A recent survey in India found that two thirds of respondents would prefer privacy over convenience when it comes to AI. These respondents aren’t necessarily against using AI but their preference is for safety. How does this disconnect arise?

A key issue here is that we don’t have feedback for AI in the same manner as most other products. When a person goes to a store they make a conscious decision to buy bread, milk, vegetables and so on. When they go to a car showroom they make a conscious decision of which car to buy. These conscious decisions tell the manufacturers of these products what they need to produce. They even tell manufacturers how they need to produce. As environmental concerns have become more widespread manufacturers are being forced to show that their goods are made sustainably. With AI on the other hand, people generally don’t make a conscious decision to use it because it’s usually designed to be invisible. Instead of explicit feedback, we have the behaviour that results from interacting with an AI product. If an interface change increases user clicks and time spent we say it was a good change. But we don’t know if the user feels better about the changed product. The way information about user attitudes tends to come out is through news stories about abuses or surveys.

Chatbots are a great example of this problem. Organisations all over the world are deploying chatbots for various reasons, arguing that they will streamline business functions and ultimately improve customer experience. An article on Yahoo reports that the chatbot market is expected to grow at an annual rate of 20.6% to reach $14 billion by 2028. And yet, a Forbes study from 2019 found that 86% of customers would rather speak with a human than a chatbot.

Users don’t have a way of expressing their choice with respect to the way AI products behave. While AI creators are required to provide controls such as letting users limit the data they share, this does not empower users to indicate that they don’t like particular aspects of a product. We need to figure out ways to get this kind of feedback so that our products are more in line with user expectations.