The Matter of Measurement

Who should you write up, who should you retain, and who should you groom for promotion? Growing companies eventually discover that they need a way to make more than just black and white distinctions between employees who are doing their job and those who should be fired for incompetence, insubordination or simple laziness.

They need?KPI’s: Key Performance Indicators. But no one is really prepared to take on this job. Even seasoned executives, HR professionals and serial entrepreneurs can get tripped up by simple pitfalls, like finding themselves incentivizing the wrong behaviors despite the best intentions.

A great benefit of being a coach and consultant is that I get to learn about brilliant things happening in business and leadership. I also learn about interesting failures and mistakes.

A favorite example of key metrics gone awry comes from the call center industry, where operators are?often measured by the number of calls per hour. When this efficiency metric becomes paramount, there are some obvious, and some less obvious, unintended consequences. All of them happened to my global call center client.

On the obvious side, operators will quickly realize that calls need to be short. So, they erroneously experience “dropped calls,” or the operator documents and declares the unresolved case closed prematurely, much to a customer’s confusion. When this “call-dumping” spreads, customers get poorer service overall. That generally undermines performance on a different metric: customer satisfaction or retention.

Another strategy that ambitious operators use is to?escalate calls to a supervisor.?

Supervisors won’t dump calls because there are additional metrics in place for them. However, calls-per-hour metrics are intended for cost containment—containment that happens through rigorously managing headcount.

But call dumping leads to unresolved customer calls. And customers whose complaints aren’t resolved call back. Ergo, call volume goes up. Systemic call dumping can mean hundreds of more calls — and the cost of the operators to handle them. Escalation also leads to higher costs. When better paid supervisors are the ones resolving low-level customer service issues, the entire value chain is inverted. Again, costs go up.

If we keep looking we’ll find more costs in customer dissatisfaction, reputation and so forth— all of which impact revenue.

A single, faulty metric like this can wreak havoc.

Technology companies have different issues but similar problems in performance assessment. A coaching client of mine worked as lead in a data science team at one of the FAANG (now MAANG) companies. He had a counterpart in the?product department, a supervisor above him and he himself managed a small team. As I worked with him, I learned about the performance assessment model.

The assessment system was a combination of a subjective 360-degree reports from the two peers and boss?and ongoing analysis of his data experiments. Those experiments are at the heart of the job. They are meant to be tests of new product features and changes to then UI. Each experiment is based on a hypothesis that it will cause an increase in a specific set of customer behaviors.

Predicting what features will drive results takes a combination of product knowledge and what might be called “reading the tea leaves” for possible trends. He was failing at both halves of the assessment.?I wondered why such a smart, committed and innovative person was doing so badly in this performance measurement process.

In analyzing the critical peer comments and their implications, we discovered the criticism was always the same?and was only given by one peer. It included?no?examples of the behavior and wasn’t echoed by the other peer. But, it became the focus of the supervisor’s attention.

When we explored the source of the criticism, the only example ever given was a single incident in his first week on the job in an interaction with his product counterpart. The team was small, so there were only three inputs in the 360 making a repeated comment statistically noticeable. It seemed a personal grudge was at play.

The bigger problem resided in the quantitative part of the assessment. His experimentation score was roughly 50% “failure”. The suggested cap was 30%. “What makes an experiment a failure?” I asked. An experiment was a failure whenever the tested feature didn’t raise the desired aspect of customer performance.

Think about that. This is a leading technology company, claiming novelty and innovation as its core differentiator. But the only way to succeed is to craft “experiments” that produce the desired outcome over 70% of the time. With only 3 out of 10 failures permitted, there is little incentive to try anything uncertain or very novel. The smart way to work if that is how you’re measured is obvious. Create experiments that are virtually indistinguishable from either existing ways the platform works, or that are lifted from competitors. If you do that, you are assured a low failure rate.

You’re also in no danger of discovering or creating anything disruptive or extraordinary.

The metric does not seem designed to incentivize innovation. It incentivizes iterating upon what has worked before. Perhaps the purpose of the metric is to encourage finding things that work! But it fails at anticipating how?employees will adjust and retrofit their own behavior to hit the mark.

These two examples both come from huge, global companies and represent trends across big swaths of the business world. They should serve as cautionary tales for companies considering their own performance measurement tools.

Ideally, before crafting any metrics beyond lagging, financial indicators, leaders should undertake serious strategic planning and develop a?strategy map?(or similar flowchart depiction) of all of the processes and phenomena driving their core business. Metrics should be tested and analyzed to make sure that they drive?the right behavior —?not just along one perspective but?along all?of them.

That means asking what each metric will cause in the life of the employee and how they will work to hit it.

For example, if the goal is sales, that’s a big-line item. What drives sales? Lead-building, follow-up, appointments, more follow-up, negotiation, deal-closing. They all need measuring, but the individual metrics don’t equal the goal, which is?new, paying and happy customers.

That’s where?Wells Fargo?went wrong. They measured accounts opened and nothing else. Employees were determined to succeed on?the only important \metric: Open New Accounts. To do that, they created accounts willy nilly, without the actual participation or consent of customers. By now, we all know how horrendous the outcome was. Customers ended up with thousands of dollars in fees and debt, and Wells Fargo was accused of and penalized for fraud.

But Wells Fargo executives hadn’t exactly “intended” to create fraudulent accounts.?They just failed to consider what a single, outcome-linked metric would cause when managers pressed employees to meet the goal of X number of new accounts; and what employees would do to keep their jobs and get their bonuses.

If only one number counts for your survival, human beings find a way to hit that number. People do what they must to succeed.

In general:

• ?Start with a?strategy.
• Develop metrics beginning from the overall desired results and?drilling down to the specifics.
• Take Perspective. Look at it from the employee’s point of view. How will they hit that number?
• Test, retest and change as needed.

If you’d like to review your strategy and metrics, schedule a call with me. We can go over what you have in place, and discuss approaches to your strategic planning process.