Are your employee experience surveys actually measuring what you think they are?

It can cost a lot of money to conduct an employee experience survey with some suppliers promising the benefits of a robustly tested set of questions and industry benchmarks. But are you getting what you think you are paying for?

As someone who has been around many different types of surveys over many years, I continue to be fascinated with the price-tag of employee surveys and software tools. Yes, the data that comes from surveys can play an important role in decision making. However the value of surveys is not necessarily in the survey itself, it is in the interpretation of the results within the broader context of what is happening in and around the organisation. In the same way that a car's instrument panel is useful but you wouldn't drive your car by just looking at the gauges, it's all the other context that makes surveys valuable, not the survey data alone.

Yet somehow, employee survey results ARE often evaluated in isolation, which can be dangerous in a number of ways. In this article, I wanted to point out a few reasons why you need to be careful with any employee survey results you receive, and make sure that just because you paid a pretty penny that it doesn't mean they are necessarily more valuable to the organisation.

Goodhart's Law

British economist Charles Goodhart is credited with the saying 'once a measure becomes a target, it ceases to be a good measure'. The rationale is that as soon as we set a specific goal, people will tend to optimise for that objective regardless of the consequences. Employee survey results are a classic for this, because KPI's are often set to encourage Leaders and Managers to take employee experience seriously. However once these measures becomes goals, it opens the door to all sorts of behaviour and manipulation to the extent that one needs to ask, are we measuring what we think we are measuring? Sometimes the answer is definitely no.

Opinions are not facts, they are opinions!

One of my long term mentors once gave me this advice, and it has stuck with me for a long time. Surveys ask for opinions, but the survey data is often treated as 'facts'. Just because someone is having a bad experience at work doesn't necessarily mean that the workplace is responsible for their bad experience. if you have ever worked with someone who tends to complain a lot, you will know what I am talking about. In fact, two people can be doing a similar job with the same boss in the same office environment and have very different opinions about their experience at work. Yet, many employee survey providers will remain adamant that they are helping you to collect facts upon which to make decisions on (which justifies the price tag). Opinions are not facts, they are opinions.

Benchmarks are rarely meaningful

Many years ago, I was one of those people who really valued the idea of benchmarks. In theory, they represent a great way to compare one 'thing' versus another 'like-thing'. In the world of surveys, benchmarks often take the form of demographic style information like company size, industry and location. This seems appealing, until you scratch the surface. Two companies can be in the same industry, be about the same size, and be located in the same places. Yet their strategies, product mix, innovation prowess and target-markets can potentially be quite different, which means the benchmarks are not really comparing like for like. In other words, two companies that seem the same can be very different, and comparisons are not all that useful. What would be really useful is to have this sort of information included in the benchmarks (strategy, target markets etc), but this rarely if ever happens because it is too hard to benchmark. Yes, ironic. Eventually, I realised that benchmarks are just a way for those who have them to charge more money, but their usefulness is relatively limited. The only real benchmark that matters is against your own strategy, otherwise you are just comparing yourself to 'average', and who wants to do that.

An approach to consider

Instead of becoming too enamoured with surveys for the purpose of measuring employee experience, one of the things you might want to try is to implement an approach similar to the one we are calling InnerIQ. In other words, to tap into the inner intelligence of your team members to uncover the very things that are frustrating them, as experts in their roles. Ironically, this is the going to be the quickest way to improve a team member's experience, and the good news is that you don't have to ask 50+ questions to achieve this. Just a handful of carefully asked questions can put you on the path towards uncovering all sorts of opportunities that you may never have even considered previously. And you might just find that the value of your typical employee survey isn't as much as you have been paying.