Decoding customer interviews like never before

How a data model and AI are revolutionizing the qualitative research process.

Solving the puzzle

To set the stage, let's step back and take a look at the challenges of customer research.

Have you conducted interviews to learn about customers? Then you probably noticed how hard it can be to see the forest for the trees. Recognizing clearly how customers want to improve their lives is difficult when it is woven into dozens of statements. The challenge is that customers often cannot or do not want to explain their behavior due to unawareness or because they don't want to reveal their inner feelings. It doesn't help that buying-stories are often full of contradictions and irrationalities (from the perspective of the interviewer).

Seeing the forest for the trees

In customer interviews, we hear many details. However, this complex and unstructured input is hard work to analyze. While activities are relatively easy to identify, understanding the purpose behind them requires a comprehensive picture. The why is typically buried within the story. The good news is that there is a good chance to recognize the motivation for taking action when we complete the puzzle.

Turning statements into puzzle pieces

When we think of customer interviews as puzzles to be solved, then the customer's statements are the puzzle pieces. By using a data model that consists of 12 types of elements (12 Elements of Customer Progress Design ("Elements")), we can turn each statement into a data point. Our model consists of factors that help us understand what making progress to customers looks like: data related to the Jobs to be done (JTBD), the context, and the forces of progress.

Each statement is put through this filter, assigned to an Element type, and captured on a sticky note. These Elements are then placed on a The Wheel of Progress? canvas.

Up until now, the statements have been assigned to the 12 Elements by note-takers. Trained and experienced interview teams were able to capture notes in real-time. This practice reduced the time and effort compared with traditional evaluation methods by factors.

Can AI help us to streamline our process?

With the emerging proliferation of artificial intelligence, we have asked ourselves how this technology can be usefully applied to our process to increase the efficiency and the quality of the result. We have seen examples of people using Generative AI to magically produce product strategies, marketing campaigns, and positioning statements. But can it also be useful for processing proprietary information gathered through customer interviews, using our data model?

Assigning the 12 element types to statements unequivocally has been challenging in some cases. But over time we were able to refine our Element criteria, which provided clarity and helped to reduce the need for interpretation. However, there are still statements in any given interview for which the assignment is not straightforward and needs to be discussed.

As human beings, we are able to deal with ambiguities. After all, we can discuss the right assignment with others and justify our decision. We can even later on change our minds. Machines don't have to ability to do that. They need to decide right on the spot. And if the AI engine wouldn't be able to produce useful output, its value would be greatly diminished.

So, we set out to try it and to develop an AI-based app that could effectively support the interview team by analyzing interviews, reliably assigning the right Elements, and phrasing them so they are useful.

Promising test results

After starting tests with our AI web app, based on real customer interviews, it quickly became clear that there is an enormous potential to support the interviewing process: an opportunity to increase the speed, accuracy, and completeness of the items captured. A recent test based on a 45 min. interview with Amélie, an electric cargo bike customer, turned out results already useful for enhancing the evaluation process.

While the quality is still a bit inconsistent across the 12 Elements, the output can already make a significant contribution to the evaluation process.

The major benefit of leveraging AI right now is the increase in completeness of the items captured. Our solution produced significantly more Elements than our manual process. With our sample cargo bike interview, our AI-based app found 180 items (including some duplicates). With our real-time evaluation, we came up with 98 items. Although the solution is not perfect yet, we will start to augment our process with our Customer Progress Design Companion (working title).

Increased trust in the data

By using our AI solution, we could also feel an increase in trust in our own findings. If AI came up with the same conclusions, then our results would be valid. Any AI solution should not be run on "autopilot". It should rather be like an assistant doing cumbersome work and serving teams. With this in mind, we will continue our solution development: improving Element identification, Element clustering & generalization, and item transfer to MURAL, and possibly other whiteboard tools.

Finally, let's take a look at what we learned from using the AI-enhanced data collection.

What progress was Amélie seeking?

After working through the data, we realized that previously we had missed some key insights. Being aware of all the "puzzle pieces" we recognized much clearer the purpose behind Amélie's cargo bike purchase. Our conclusion was that she aspired to be the best human being she possibly could be: A good mom to her toddler and unborn baby, a good wife, daughter, and daughter-in-law, and be respectful of other people and nature.

This customer job at the highest altitude drove her to seek and utilize a new mobility solution: one that reduces her dependence on others and unreliable public transportation, eliminates her fear of driving, and safely transports her children while reducing her carbon footprint. Awareness of these customer jobs can now help to guide purposeful strategy development.

Will we train ChatGPT with our data?

One question that comes to mind when considering using an AI solution is whether my proprietary data will be used to train it.

The protection of our clients' data and investment in customer research has been a prerequisite for even considering an AI solution. It is of our utmost importance. To ensure data privacy, all OpenAI responses are only shown to the user and never stored in the system.

Also, OpenAI states publicly that they do not train their models through input and output through their API. You can read the documentation at https://openai.com/api-data-privacy

Conclusion

The decoding of customer interviews is key to meaningful insights. It gets much quicker and easier with a data model. It becomes even more easy if you have access to the full wealth of structured information from the interview. Finding the purpose in purchase stories and obtaining other data related to the buying process is key to developing meaningful products, marketing campaigns, and buying aids.

AI will become sooner or later a valuable work aid for customer researchers, supporting this process. Our desire is to be amongst the pioneers of structuring qualitative data and help our clients provide breakthrough insights: Insights that support strategy development that help people make progress.