Be careful what you ask for...

Designing an employee engagement survey is easy.

Designing an effective employee engagement survey is harder.

And making long-lasting change happen based upon the results of an employee engagement survey is really, really hard.

But there’s one area of an employee engagement survey that is particularly risky: open-ended comments.

Right here, I’m going to state a strong preference - when it comes to open-ended comments, it’s better not to ask. Just don’t do it.

Before I tell you my two main reasons, let me share an experience that sits deep in my thinking here.

Long before I inherited the Pfizer Global Colleague Survey, and while I was still back in the UK, I was involved in revamping our sourcing and selection activity, which included submissions to benchmarks like the Great Places To Work list. As a natural culture hound, I’d been asked to join several teams, including a local survey. The US was parallel tracking and so I visited our NY HQ on a fact-finding mission (basically to ask whether the then US-based survey was expected to go global in the short- to mid-term).

It was a great visit but there was a really disheartening moment. 

At one point, I was left to my own devices, and my host pointed out a room:

“You can read the comments if you like…”

So, being of an inquisitive nature, I walked into the room. 

And there was what I call THE WALL OF DEAFENING SILENCE.

Six shelves high, wall-to-wall, binder upon binder of printed comments from the employee survey. As I pulled samples, I noticed dust on the covers. Many of the pages were pristine, and even stuck together slightly.

Now, I don’t know if I can say those comments had never been read, but to this day, I’m pretty sure they hadn’t been read recently.

(bear in mind, this was LONG before AI, or even integrated databases with search and text-aggregation)

Now, I knew that the survey was well-intended: 

“We want to hear from you...”

and the response was given honestly:

“We want to be heard..."

But reality often doesn’t match the intent - and too often survey comments end up going into a black hole of 

“I’ll get around to reading them…”

In fact, because executives are busy, more often a junior employee or consultant is asked to summarize for themes - which totally and utterly gives lie to 

“Go ahead, I’m listening…”

(and, by the way, how long until the comments lose validity and reflect history rather than current state?)

Now, I have to say, when I first ran the Global R&D People Survey, we included 3 open-ended questions 

(basically Start/Stop/Continue)

and I know of at least one executive who read all 2,000 pages of comments

(though she admitted it was tough going)

Sometimes, someone is listening!

Still, this experience provides my first reason for not seeking comments: 

By asking for open comments, you run the risk that your actions won’t match your words, and you will erode trust

Before we place this all on the shoulders of executives though, let’s flip the lens over and look at who’s commenting.

Now, first things first, if your survey is completely confidential

(and it should be for engagement data)

then you don’t KNOW who’s commenting. 

And that means you run the risk of transferring your own meaning and prejudices on the comments that are written.

For example, comments such as 

“HR should deal with the performance issues around here”

is interpreted as criticism of the HR function and the performance management process

(spinning up workstreams, and all manner of activity)

when in fact the comment MAY reflect a very specific issue with a Manager playing favorites and cronyism at the local level.

And was this the comment of a Manager who needs more help, or a high performer who’s having all the work dumped on them because others in the team aren’t pulling their weight.

It may be any or all of the above - it may be any number of other scenarios.

The fact is, we just DON’T KNOW.

And I’ve sat in rooms with leaders discussing comments alongside other themes in the data only to hear:

“But what did they MEAN when they said that?"

Which leads to reason two:

Seeking comments in a confidential survey will only leave you with unanswered questions

And there’s one more reason. And I’ll keep it brief because this is going a little long.

We all know how confidential commenting on the internet leads to rich, respectful, meaningful and productive conversations, right?

Right?

There’s reason three, right there!

So, what to do?

Well, we have to look at the reason for running the survey. 

There are three main contenders. We run a survey to:

1. Provide executive leadership with a “view from the bridge” of employee engagement
2. Provide front-line managers with local data upon which they can enact change
3. Provide employees with an opportunity to reflect on their experience and share their opinion

Now, while all surveys provide all three, one will be primary dependent upon prevailing organization culture

(which, of course, also means that this may be spoken or unspoken, and/or a blind spot)

In a primarily executive-focused survey, by all means add comments, and do all the necessary footwork to assess themes, etc. As you do so, though, remain aware that the best you can hope for is incomplete, fuzzy information. There may be good soundbites and, if you’re lucky, significant and consistent themes/patterns upon which executives can decided necessary strategic responses.

Maybe.

But if your culture is exhibiting any amount of executive disconnection or distrust, if your leaders are criticized for being too aloof and cut-off, beware seeking open-ended comments.

It’s in the latter two reasons for surveying that comments become truly problematic.

If we want employees to step-up and managers to take action then confidential comments don’t help.

I have been asked on many occasions to facilitate discussions between manager and team based upon comments received by the overall survey. Comments that do NOT reflect the reality of day-to-day for either manager or team in question. 

This becomes an exercise in interpretation of hypotheticals and, quite frankly, a waste of everyone's time

(in each facilitation, my approach is to encourage the manager to ask the question again and engage in a live feedback of local reality)

In my experience, it’s often the strongest managers who do the best job following up on survey comments - discussing the comments which seem to have been made by those working for the weakest managers. 

And, on the flip-side, the very managers who NEED the most change based on the comments are the LEAST LIKELY to be addressing them.

The added pain here is that over successive iterations of the survey people are going to be less and less willing to provide thoughtful, honest responses to the survey.

So, no comments?

I started by expressing a preference based on experience. No comments.

When it comes to employee engagement, the manager makes the difference. And open comments don’t serve the manager who wants to make a difference.

At best open comments offer an indicator of potential issues and/or direction for executive leadership to consider. More often, they are little than a comfort blanket for the well-intended “We’re listening…”

So, when it comes to employee engagement surveys, it really is a case of be careful what you ask…

* * *

It seems odd to ask for comments after writing all that, but I am - and I promise I will read and respond to every one! :)

So, tell me about the best and the worst of your experiences with survey comments?