Insight technology is the poster child for digital transformation

In order to stay competitive you need to understand what your customers and employees think. Finding out what the customer thinks sounds easy. You just send them a survey, right? The problem is that you have to guess which questions to ask.

One common approach is let the respondents indicate how much the respondents agree or disagree with whatever statement you came up with in your qualified guesswork. This approach has remained unchanged for a very long time.

If you are going to list all the potential things the customer could be concerned about, the survey will be ridiculously long, and people won’t answer it. And chances are that you still haven’t asked the right question.

So instead of guessing what the customer thinks you could just ask one single question: “How did you experience your flight?” or "How did you experience your visit to our online store?” or one similar open-ended question. You get will top-of-mind ("system 1") answers, without framing, or other biases, and you get much better response rates.

The problem is how to analyze this. If you have a few hundred answers, one person could read through them and summarize. But what if you have thousands and thousands of answers each month?

It is in the larger amounts of that you can get insights that statistically robust. And with larger amounts of free text answers, you get more granular and detailed insights, and which are statistically robust.

If you want insights that are both detailed and reliable, you need large amounts of data. In fact, you need data in large amounts that are uneconomical to analyze manually. In many cases, responses to open-ended questions are left unanalyzed. So, here was the perfect case for automation.

Various methods have been tried when it comes to automating the analysis of responses to open-ended questions, or of text data in general. But it turned out to be a surprisingly complex problem to solve.

Over the last few years, there has been significant advances, and modern qual-to-quant tools are now being used for analyzing massive amounts of data: not only responses to open-ended survey questions, but all types of customer interactions, such as emails, support tickets, voice-to-text data, chat dialogue, social data, streaming data, and so on and forth.

The qualitative data (the text data) tends to be quantified and and analyzed together with relevant meta data, in order to get more valuable results. But, despite these recent advances in qual-to-quant technology, some industries still run separate analytical departments for their qualitative and their quantitative data.

As insight tools are getting better, faster, cheaper and much easier to use, end users no longer need a data scientist to get deep insights from hundreds of thousands of customer feedback texts. They can do it themselves in a matter of minutes.

AI and machine learning has provided a new generation of insights technology with tremendous capabilities. New technology, in the shape of DIY insight tools, is disrupting business models for incumbents in the market research industry and elsewhere.

This new technology is now starting to change the competitive landscape across all consumer industries. The most interesting digital transformation cases, with the highest ROI, are those where insights - into what customers think - lead to continuous improvement and increased profitability.