What are the limitations of using surveys to measure escalation impact on CX?

Survey is all about timing and emotion. Knowing when to send it is critical, yet the response can truly vary unless the customer is in the rational emotional stage. To gauge the sentiment, the survey is best done after the transaction has happened or straight after the escalation resolution and taking the pulse again 3 months later. This follow up will reduce the overall impact of escalation on CX.

Respondent sample bias: In addition to response bias, more often than not, only the currently 'active' or 'pulsating' consumer base participates in surveys because they are more attached to / have experienced the brand/service in the recent past. Other 'dormant' consumers often do not respond (because hey! who has all the time in the world to keep filling surveys!). This often culminates into a biased survey read as opinions of those who are satisfied (but lazy to respond to surveys) are often missed.

The recency bias along with who receives/responds and their emotional state are the primary limiters for surveys not serving the purpose of CX management. Though these stand true to a large extent, when we look at it from a customer perspective - the periodic/milestone/transactional surveys are sent based on the sender's requirements and does not focus on providing a direct channel for customers to provide feedback when they really want to be heard. Additionally, internal stakeholders armed with the survey timing and potential target audience put in inorganic efforts to delight selective customers before the survey deadline leading to distortion in the survey results negating the intended objective.

Response bias can significantly undermine a business by distorting the accuracy and reliability of the data collected through surveys, customer feedback, and market research. When individuals do this, their feedback may not represent their true opinions or behaviors - sometimes customers feel like they're evaluating a person, and not the service. This can lead to flawed decision-making processes within the organization, as the skewed data may guide choices that do not align with the actual needs of the target audience.

The bigger challenge with surveys is that you have a very fixed set of questions to demand answers from. 90% of the time it hovers around what the surveyor wants to understand or listen from customers. CX cannot be done via a survey it is the feedback mechanism and escalations that one receives should be engaging enough to understand what is grossly wrong

While surveys can offer additional perspective, following a customers journey across key moments that matter offers strong insights to escalation points - when and where things start to break down that then leads to escalation. Getting to the root cause is key.

It’s also very important when defining timing to use data to accurately predict when the customer is most active on social media and other digital channels as this time will be most likely the best time to meet a receptive customer. The survey must be dynamic and not one size fit all when it comes to being deployed.

Important to understand objectives when designing the timing of surveys, understand overall brand perceptions? customer effort? customer satisfaction with an interaction? understanding the objective and how you plan to operationalise, is first step prior to design.

Customers may struggle to accurately recall details of their experience, especially if the survey is sent after a considerable amount of time has passed since the escalation. This can lead to memory bias, where customers may overestimate or underestimate the severity of the issue or their emotional state during the escalation.

In my experience, NPS surveys are always backward-looking tool in an era where customers expect near real-time remedies. We've seen higher impact outcomes like impacting revenue through other methods that evoke near-realtime customer interventions.

Agree. Customer stress can be mistaken for anger. Customers can become stressed by confusing rules, after spending too much time on hold, or being passed around for answers. See customers' emotions their way to earn praise and trust.

A qualification system must be in place to distinguish between different topics of escalation and allow comparability of result in the survey. Qualifying different categories of customer requests and tailoring the questions asked and timing of the survey is key to comparing the efficiency of escalation for various use-cases. Example: escalations for a missing parcel are experienced differently when the parcel contains a decorative item as opposed to a wedding ring. The answers to the survey are not comparable between these use-cases as they come with different timeline expectations and emotional loads, thus surveys must be tailored to the use-case explored.

Also, customers' circumstances and perspectives can change over time. The urgency or importance they initially attributed to the escalation might alter as days or weeks pass. Therefore, the timing of the survey may not align with their current viewpoint, leading to a mismatch in feedback.

Surveys often use standardized scales or question formats, which may not be flexible enough to capture the unique aspects of an escalation experience. Customers should literally be allowed to express their emotions and or experience(s) to better understand the intent.

While considering diverse data sources, integrating them is crucial for a holistic understanding of escalation impact on CX. It’s like assembling a puzzle – each data piece contributes to the bigger picture. By breaking down data siloes and aligning insights with the customer journey, you can uncover valuable connections and deliver more effective solutions.

Just like surveys can be effected by bias, sentiment analysis are also not 100% accurate and relevant data regarding the actual escalation can be hard to come by. And looking at behavior alone is not really a good measure of how the customer feels about the escalation itself. Here’s why: A customer that stays on as customer might do so without being happy with the escalation process or outcome . There might be a strong relationship to start with or there can be a lack of alternatives making the customer to stay. There can also be a long lag between the escalation and the customer leaving or returning/receiving and during that time a lot of other things can happen that can effect the customer behavior.

Use a solid VOC platform that utilises natural language processing and sentiment analysis techniques to evaluate customer feedback from various sources like social media, online reviews, or support tickets etc.

An another example of being able to define the impact of the escalations on the overall CX would be to look over a period of 6 months the churn ratio for the affected customers as well as the transaction frequency and value. This will give you a true reflection of the escalations impact on CX.

We started analyzing what we've found to be our largest and most valuable data source, what our customers are actually saying in the every day conversations they are having with our teams (phone calls, emails, tickets, and chats). This has yielded insights far beyond just sentiment and when combined with traditional surveys and static telemetry reports gives us the 360 view of our customer with real context so we can create personalized intervention strategies.

Response bias may distort results, as only a fraction of escalated customers respond. Surveys offer limited context, making it difficult to understand the nuances of the escalation process fully. Real-time insights are lacking, and surveys may not capture the emotional impact effectively. Additionally, they struggle to quantify long-term effects on loyalty and retention. To mitigate these limitations, it's crucial to supplement survey data with real-time feedback, customer interactions, and social media monitoring, ensuring a more comprehensive understanding of how escalation affects CX. This approach allows for informed decision-making and continuous improvemen

Another commonly overlooked limitation of surveys is Emotional Amplification: Escalations, by their very nature, often occur when emotions are already heightened. A customer might be frustrated, disappointed, or angry when they seek escalation. In such scenarios, their emotional state can amplify their response, either positive or negative, in the survey. This emotional amplification can distort the actual impact of escalation on CX, as the responses might be more extreme due to the emotional state of the customer at the time of the survey.

In my experience, surveys can be valuable tools for businesses to gather customer feedback and insights. It is essential to remember that these surveys should not be seen as a replacement for real, face-to-face interaction with customers. There is simply no substitute for the level of understanding and insight gained from genuine human interaction. When businesses engage with customers directly, they can build stronger relationships and better understand their needs and preferences. While CX surveys can be useful, it's essential to remember that they should be seen as just one tool in a more significant customer engagement strategy.