Measurements Matter

This article is adapted from one first published at?Forbes.com?on August 6th, 2018.

Who should you write up, who should you retain, and who should you groom for promotion? Growing companies eventually discover that they need a way to make more than just black and white distinctions between employees who are doing their job and those who should be fired for incompetence, insubordination or simple laziness. They need?KPI’s: Key Performance Indicators. But no one is really prepared to take on this job. Even seasoned executives, HR professionals and serial entrepreneurs can get tripped up by simple pitfalls, like finding themselves incentivizing the wrong behaviors despite the best intentions.

A great benefit of being a coach and consultant is that I get to learn about brilliant things happening in business and leadership. I also learn about interesting failures and mistakes.

A favorite example of key metrics gone awry comes from the call center industry, where operators are?often measured by the number of calls per hour. When this efficiency metric becomes paramount, there are some obvious, and some less obvious, unintended consequences. All of them happened to my global call center client.

On the obvious side, operators will quickly realize that calls need to be short. So, they erroneously experience “dropped calls,” or the operator documents and declares the unresolved case closed prematurely, much to a customer’s confusion. When this “call-dumping” spreads, customers get poorer service overall.

Another strategy that ambitious operators use is to?escalate calls to a supervisor.?Supervisors won’t dump calls because there are additional metrics in place for them.

However, calls-per-hour metrics are intended for cost containment, keeping headcount low. But call dumping leads to unresolved customer calls, ergo, they often call back. Call volume goes up. Systemic call dumping can mean hundreds of more calls — and the cost of the operators to handle them. Escalation also leads to higher costs. Well-paid supervisors ultimately make calls more expensive. Again, costs go up.

If we keep looking we’ll find more costs in customer dissatisfaction, reputation and so forth, which all impact revenue. A single, faulty metric like this can wreak havoc.

Technology companies have different issues but similar problems in performance assessment. A coaching client of mine worked as lead in a data science team at a well-known, Fortune 50 company. She had a counterpart in the?product department, a supervisor above her and a small team she managed. As I worked with my client, Elaine, I learned about the performance assessment model.

The assessment system was a combination of a subjective 360-degree survey of her two peers and boss?and ongoing analysis of her data experiments. Such experiments are tests of new product features hypothesized to cause an increase in a specific set of customer behaviors.

Predicting what features will drive results takes a combination of product knowledge and what might be called “reading the tea leaves” for possible trends. Elaine was failing at both halves of the assessment.?I wondered why such a bright, insightful and polite person was failing her assessments.?

In analyzing the critical peer comments and their implications, we discovered the criticism was always the same?and was only given by one peer. It included?no?examples of the behavior and wasn’t echoed by the other peer. But, it became the focus of the supervisor’s attention.

When we explored the source of the criticism, the only example ever given was a single incident in Elaine’s first week on the job in an interaction with her counterpart in product. The team was small, so there were only three inputs in the 360, making a repeated comment statistically noticeable. It seemed a personal grudge was torpedoing Elaine.

The bigger problem resided in the quantitative part of the assessment. Elaine had a score for her experimentation reflecting over 50% “failure” with a suggested cap of 30%. “What makes an experiment a failure?” I asked. An experiment was a failure whenever the tested feature didn’t raise the desired aspect of customer performance.

Think about that. This is a leading technology company, claiming novelty and innovation as its core differentiator. But the only way to succeed is to craft “experiments” that always produce the desired outcome. That isn’t experimentation and it definitely doesn’t lead to innovation.

It seems to disincentivize experimentation —the source of real innovation. To succeed in hitting the goal, one would need to rig the system of predicting customer behavior with new features. How do you do that?

The answer is sadly obvious: You repeat what has worked before. You copy and repackage features from the past, from other products in the company or from competitors. That way, most?of your experiments are “successful.” But at what cost? You can see that the purpose of the metric is to encourage finding things that work! But it doesn’t address?how?employees will adjust and retrofit their own behavior to hit the mark.

These two examples both come from huge, global companies and represent trends across big swaths of the business world. They should serve as cautionary tales for companies considering their own performance measurement tools.

Ideally, before crafting any metrics beyond lagging, financial indicators, leaders should undertake serious strategic planning and develop a?strategy map?(or similar flowchart depiction) of all of the processes and phenomena driving their core business. Metrics should be tested and analyzed to make sure that they drive?the right behavior —?not just along one perspective but?all?of them. That means asking what each metric will cause in the life of the employee and how they will work to hit it.

For example, if the goal is sales, that’s a big-line item. What drives sales? Lead-building, follow-up, appointments, more follow-up, negotiation, deal-closing. They all need measuring, but the individual metrics don’t equal the goal, which is?new, paying and happy customers.

That’s where?Wells Fargo?went wrong. They measured accounts opened and nothing else. Employees were determined to succeed on?the only key?metric: Open New Accounts. To do that, they created accounts willy nilly, without the actual participation or consent of customers. By now, we all know how horrendous the outcome was. Customer ended up with thousands of dollars in fees and debt, and Wells Fargo was accused of and penalized for fraud.

But Wells Fargo executives hadn’t exactly “intended” to create fraudulent accounts.?They just failed to consider what a single, outcome-linked metric would cause when employees adjusted their own behavior to keep their jobs.

Without metrics that measure more than the big number,?people do what they must to succeed.

It’s the difference between playing the game well and only playing for the win.

In general:

??Start with a?strategy.

? Develop metrics beginning from the overall desired results and?drilling down to the specifics.

? Look at it from the employee’s perspective.

? Test, retest and change as needed.

If you want metrics that are connected to a real, testable, breakthrough strategy, schedule a call with me. Beyond Better offers world-class strategy consulting that will catalyze your own brilliance to accelerate your growth!