The survey is dead! Long live the survey!

“With all the information we have, can’t we figure out what a customer is going to do next without asking them?”

This is the question every CX manager grapples with every time they evaluate a Voice-Of-the-Customer (VOC) program. It does make sense. With omni-channel customer interactions via chats, phone, web app, mobile app, and CRM, do we really need to send a survey to customers?

A little bit of history

A VOC program where the customer is asked about their experience has been around for a while. In 1994, the National Quality Research Center (NQRC) at the University of Michigan's Ross School of Business established the American Customer Satisfaction Index (ACSI). The ACSI was the first national measure (in the USA) of quality in goods and services from the perspective of the customer. Over the next decade, CSAT surveys dominated the landscape. Market research organizations setup dedicated call centers that would call the customer to solicit feedback. These projects would run into millions of dollars and produce binders of reports that companies would then try to decipher to take corrective action.

Then came CRM and it changed the customer game overnight. Suddenly, the customer wasn't an esoteric, exotic being but someone who the company was able to understand at a granular level. The customer became connectable, within reach! Then, the VOC program changed as well. Satmetrix Inc. and Bain & Co. jointly created the NPS? and published an article on HBR that revolutionised the VOC market. Now, every customer was assessable using real-time surveys. Feedback was no longer sought from a few, but from everyone who interacted with company. The biggest attraction for the NPS? was the connection it established, for the first time, between customer experience and financial outcome.

This was a BIG deal! For the first time, companies started to see a real return on the investments they made in improving customer experiences. This started a change in the way people perceived VOC data. Where earlier, there was a 'belief' that customers' experience resulted in positive financial outcome, now there was proof that it did. Industry leaders were quick to seize this advantage. We saw companies realign their entire operations, starting from the front-line to the top management by incorporating the NPS? metric in daily functions and decision making.

Today, the most evolved companies are making sure that every customer feedback is taken into consideration when building a customer focused team. However, only 13% of CEOs are confident that their organization can take action on the data in the near future.

This is because VOC alone is not sufficient to make predictive decisions for the entire customer base. Hence, the ROI models are difficult to make sense of and make it less compelling to double down on Customer Experience (CX)

The future of CX

Customer experience (CX) considers everything the customer goes through—it’s everything the customer touches, tastes, smells, hears, sees throughout the experience with the brand. What are they going to feel or think? It’s being almost obsessive about the experience the customer has with the brand—the attitude of ‘I want to be here,’ rather than ‘I have to be here. - Blake Morgan, Author, Forbes columnist

There is growing belief that the wealth of information generated through the many interactions a customer has with a company should be leveraged to create models that predict the financial outcome from individual customers. The holy grail today is to predict the customers' future behaviour using their historic behaviour and attitudes.

Customer Data Lake

The first step in this process is to create "data lakes" that record the entire customer journey. Imagine the customer as he interacts with the company. He buys a product, calls customer support and chats with an agent, visits a store, answers a survey, returns a product giving a reason etc. Each of these journey points generate a small amount of data that goes into the data lake.

As more and more such drops of data start filling up the lake, the entire customer base is slowly represented this data. There are mainly 3 types of data being generated.

A. Behavioral data

This is data that is generated by a company's CRM/POS/ERP systems. For example, a customer buys a product from a store and gives their loyalty card details. The POS system captures the customer ID and associates the purchase details, like shop ID, item ID, sales price, discount, sales rep ID, etc. Later, this same person might call the support center to raise a complaint. The chat session (open text) will be associated with the customer ID and support representative ID, and stored in the lake.

B. Conversational textual data

All organizations are speaking with the customer using multiple channels. For example, the customer can speak with the company using a chat (bot) interface. They would be speaking to customer support representatives, which would generate a transcript of these conversations. Some of the customers would also be filling out surveys, which also generate text data. Additionally, via social media, there would be additional conversations/comments associated with the customer as they voice their opinions or questions.

C. Attitudinal data

This is data generated mostly from surveys and feedback collected across the organization. It allows the customer to objectively rate the interactions they have. These data sets can be from different sources. Sometimes, the data will be associated at a customer level, and other time, the data is not identifiable at an individual customer level. For example, there are studies that measure the relative 'Awareness' or 'Consideration' of a brand when a person makes a purchase decision. These kinds of studies are not done just for the company's own customers. Opinions are sought from customers across the market. Data from these sources are also stored in the data lake and included in the next step.

Building the Predictive Models

Once all this data is in the system, the next step is to get these disparate data to speak to each other. Here, a team of data scientists builds the models required for a specific company (and every company is slightly different). AI is used extensively in this step where the data is ingested by these system. By observing the actual behaviour of the customer (churn out, new purchase, reference etc.) and modelling that against the inputs received from the data sets above, models that predict future customer behaviour start to emerge. These models are extensively tested and tweaked to increase accuracy. Once ready, these models are integrated with the pipelines of the CX systems so that everyone in the company can access these outputs. 

Personalized experience for each customer. 

Now the company is ready to personalize every customer interaction. Say, over time, the customer calls the call centre, and expresses his dissatisfaction. The system knows that this will probably lead to an exit event and prompts the customer recovery team to make a proactive call to the customer to offer some benefits commensurate with the customer lifetime value. 

Surveys make the predictive models far more accurate than otherwise possible.

Only 7-8% of customers respond to a survey. So, how do they make the model better?

See, the magic of surveys is that they quantify attitudinal measures and outcomes. When we add an NPS? or CSAT metric into the mix, we are able to quantify ‘customer emotion’ in a discrete range of values. This is key in any model building. Even if only 8% of the respondents answer a survey, we are able to use those respondents as anchor points in a predictive model that makes the rest of the model stronger. 

Let's consider a project NUMR recently did for an airline company. We had the following sources of information in the data lake:

• Post taking a flight the customer answered an NPS? question along with an open ended text question. [ ~8% of customers]
• Customer support call data (transcribed) [ <1% of customers]
• Booking details (place, time, medium etc.) [100% of customers]
• Flight operations (origin, destination, on-time, delayed, international, domestic etc.) [100% of flights]
• Customer loyalty program details (loyalty points, number of flights take by this company in the past 12 months, proffered origin, preferred destinations etc.) [100% of customers]

What we were trying to predict is this- would the customer use our client's airline the next time they book a flight?

We were initially only working with the data sets where we had 100% of the data. The hypothesis was- since this information is available for all customer, this should be sufficient to predict customer behaviour. We ran the models and the best case predictability (for those who had taken more than 4 flights in the last 3 months) came to 66%.

We then incorporated the surveys and the chat sessions. The chat sessions generated a lot of text data. When there is a lot of text generated, the ability to attribute the right amount of weightage to the right context is a difficult task. For example, say a person is making a booking and while chatting with the representative, they ask for a layover break. The representative explains that it is not possible for this class and the conversation moves on. There was no sentiment that hit the required thresholds and hence no significant variation was observed. This, along with the low volume of people who called support, did not add much to the models. So, including chat sentiments in the model improved the model slightly, but not that much.

However, when we incorporated survey data, the picture changed. Now we had a hard outcome based on the items people were speaking about. By analyzing the text and then associating it with flight and customer details, we were able to generate a model that allowed us to predict with 

·      85% accuracy for those who answered the survey and,

·      With 74% accuracy for those who didn't. 

For those who didn't answer the survey, the accuracy improved as we were able to give appropriate weights to the parameters for flight and customer details that were undiscovered prior to including survey data.

In Conclusion

Let’s come back to original question-  do we need to have the survey at all? 

The answer is 'Yes' if one wants to improve the predictability of the models. Customer opinions and attitudes are best understood when the customer directly says it. Of course, one cannot only depend on surveys as they only address a few customers. But once this is included in the predictive model and as part of the CX platform, the effectiveness of one's CX initiatives increases exponentially.