The Future Of Customer Experience Is Here: Are You Ready?

Authentic Emotion Recognition Can Change Your CX for the Better

In the Olympics, who's happier: the person who receives the silver medal or the person who gets the bronze? You would probably say the silver medalist, who came in second instead of third, will be happier because objectively, it makes sense.

The answer, which I will reveal in a minute, is the reason it is paramount that you measure customer emotion during your Customer Experience in real time. 

Our guest Dr. Bill Hedgecock, associate professor of marketing Carlson School of Management at the University of Minnesota, knows the answer. He joined us on a recent podcast to talk about how we can measure emotion in our digital world to understand how customers feel in our experiences. 

As a leader on the forefront of facial recognition technology and facial expression analysis within the academic marketing community, Hedgecock does psychology-based research and studies neuromarketing, or how the brain works when we make decisions. Hedgecock says one of the values of using this kind of technology is we understand why people have specific reactions or behave a certain way. 

If you consider customer behavior in a Customer Experience, we intuitively think that people are going to be happier when they have a better outcome than a worse outcome. For example, if your customer receives a package in two days, they're probably more satisfied than if they got it in three because we want things delivered faster and are disappointed if it takes longer. Another example might be if people are waiting in your store for three minutes to check out, they're probably less happy than someone who waited one minute. These things make sense to us.

However, this logic does not always hold true, especially when you consider Olympic medalists. Hedgecock did some research on the facial expressions of Olympic medalists that disproves this concept. His research suggested that silver medalists were less happy than bronze.

For example, consider the facial expression on Silver Medalist McKayla Maroney, an Olympic gymnast from the London Olympic Games:

To be fair, you don’t need facial recognition software to figure out how she feels here. 

Now, she took a lot of criticism about this moment, but how she feels is understandable. She was so close to winning the gold that getting silver feels terrible. The bronze medal winner, by comparison, is ecstatic. They realize they could be out in the audience with the fourth-place finisher. At least bronze puts you on the podium. 

However, neither medalist is as happy as the gold medal winner.

This example demonstrates the concept of Reference Points in action. Moroney’s silver feels worse than bronze because it was so close to gold, but not close enough. Silver medalists were almost the best in the world. Also, to Hedgecock’s point, this example shows how an objectively better outcome made someone less happy than a worse one. 

Now, when I explained it, many of you understand this outcome right away with the Olympic medalists. You may be familiar with the phrase, “second is just the first loser.” Maybe you heard it from an over-zealous coach in your past or a from the clever character in a movie, but we are all familiar with the go-big-or-go-home mentality to sports achievement. However, this behavior spills over into our non-Olympic-medal-winning lives also. 

For example, let’s say I'm standing in line at Starbucks, and I expect it to take two or three minutes to get my coffee. If it takes a minute, I am happy because it went faster than I thought. However, in some other store, not Starbucks, if I go in and I wait just two minutes, I might be outraged because I think I should have been done already. 

In other words, waiting two minutes is fine until it isn’t. 

Our job as Customer Experience consultants is to determine when it’s fine and when it isn’t. In the past, we relied on research like surveys to help us understand how “fine” our Customer Experience is. 

Unfortunately, surveys aren’t as accurate as  you might think because it is after the fact, and there are few different influences on how people remember things. Facial recognition technology and facial expression analysis can help us capture more accurate data about how people feel because it is happening at the moment. 

For their research, Hedgecock and his team pulled photos of the last five years of the Summer Olympics. They compared the expressions of the medalists. They discovered some exciting nuances between the facial expressions of the winners. One of them was about the authenticity of a smile. 

When people fake a smile, you can usually tell. Sure, their teeth are showing, and at first glance, they appear to be genuine in their happiness. However, something doesn’t feel right—and it usually stems from the eyes. 

The muscles around the eyes contract when we smile. Very few people can fake this effect; it is involuntary. However, Hedgecock says a few people can, like models. 

In the context of the Olympics, the research team noticed that while the Gold Medalist was beaming with joy, often the bronze winner was not. Sure, they were smiling, but not with the same intensity as the gold medalists, understandably. 

Hedgecock says his research shows how when you combine the theories of behavioral economics with technology, you get powerful data that allows you to spot significant patterns in customer behavior. You often have two to three theories about why people do what they do, and any one of them could be right. However, if you have data on facial expression and the facial expression the customer has associates them with one of the three theories, now you know why people did what they do, and you can respond. In other words, it can solidify your conclusion.

Emotions drive our customer behavior. It also shapes our opinions of a brand or an organization. Our feelings influence our thinking, which, in turn, drives our decision making, which can be in or out of your favor. 

The influence of emotions on your Customer Experience cannot be understated. It is essential to know how what you are doing to customers in your interactions makes them feel, whether online or in-person. 

The implications of this information are staggering. Imagine if you could use the information about how customers feel to determine how people think in real time without asking them. After all, people often don’t want to tell you, maybe because of the time it takes or perhaps because they are embarrassed by their genuine emotions. 

Also, the act of asking people how they feel about an experience changes their answer. When we conduct market research of any kind, we encourage a different state of mind than customers would be otherwise. After all, you don’t think about an ad the same way after sitting through a focus group where you talked about it for an hour. Capturing people's automatic, real-time reaction is better and more useful data than traditional methods of market research. 

Moreover, asking people how they feel about an experience isn’t as easy for them as you might think. Often, people don’t know how to tell you how they feel. Verbalizing emotions is not a skill that comes to everyone naturally, and for some, it is next to impossible.  

Also, your emotions change as you progress throughout the interaction. Can you think of a time when you were angry about something as a customer, went to customer service where the company clarified the misunderstanding and then you were okay? Me too. Of course, if you get the wrong employee, or, for you, it’s the wrong time of day or even the wrong day altogether, that can also go the other way.

Genuine customer emotion measurement bypasses all of these variables and influences and helps you understand how your Customer Experience makes people feel when they are feeling it. Most importantly, you know what you need to change in the experience to improve your results, i.e., make customers feel the way you intended. 

Facial expression analysis using facial recognition technology is a fascinating subject.  Using technology to gather how people feel is where Customer Experience is heading over the next few years.  We are all on the ground floor of understanding it. In our Global Customer Experience consultancy, our latest training explores how to use it to improve the impact (Read: ROI) of your marketing efforts. 

Olympic medalists can teach us a lot about how facial recognition technology and facial expression analysis captures what a person is feeling in real time. Having this data in real life and associated with our brand and Customer Experience is critical to taking your experience to the next level. When it comes to Customer Experience design, having technology that helps you understand all the complexities of human emotion in interaction is the type of thing that can help you take your experience to the next level, and go for the big-grin-wearing-eyes-smiling victorious gold.

To hear more about measuring emotion in our digital world in more detail, listen to the complete podcast here. 

Measuring authentic customer emotions in real time without asking customers is the future of CX. Understanding how your experience makes customers feel in your experience shows you what you need to keep doing—and what you need to change. Moreover, you can improve the impact and perhaps most importantly the ROI of your marketing content. To learn more about our New Authentic Emotion Measurement Training, please click here. 

Hear the rest of the conversation on Measuring Emotion in Our Digital World on The Intuitive Customer Podcast. These informative podcasts are designed to expand on the psychological ideas behind understanding customer behavior. To listen in, please click here.

Colin Shaw is the founder and CEO of Beyond Philosophy, one of the world’s leading Customer experience consultancy & training organizations. Colin is an international author of six bestselling books and an engaging keynote speaker.

Follow Colin Shaw on Twitter @ColinShaw_CX