Crappy metrics are killing Software Teams

TLDR; Org metrics fine. Individual metrics bad. Psychological Safety best.

Many technology companies have become obsessed with quantifying everything. Lines of code, pull requests, story points, bugs fixed - whatever it is, someone out there is trying to measure it. These metrics are killing teams, stifling innovation, and creating a culture of mistrust.

How did we get here?

The curse of the good engineer

Engineers are hardwired to love measurable outcomes. There's a dopamine hit when we see a compiler spit out "0 errors" or watch all our tests pass, or create 30 new type definitions so that our 10 line file is now "safe". For the most part, software engineering is about defining the world in a simple manner. The best engineers excel at doing so, and we reward them with a promotion to a completely different job that they probably suck at.

Managing people isn't managing code. You can't compile a team and get an error report. Human productivity is messy, nuanced, and often invisible to the naked eye. It's also a totally unsolved problem. Unsolved like "everyone tried to solve this and failed", not "the world has been waiting for you to solve this".

Many engineering managers find themselves in a peculiar position. They've spent years honing their technical skills, where every problem had a measurable solution. Now, thrust into leadership, they're faced with challenges that defy easy quantification.

People be complicated, yo

Focusing too heavily on easily measurable metrics creates a host of problems that cripple talented teams. One fantastic source I can recommend about this topic is Measuring and Managing Performance in Organizations by Robert Austin .

Intangibles

Some of the most valuable contributions to a team don't show up in any metric:

• Mentoring junior developers
• Improving team morale
• Writing clear, comprehensive documentation
• Fixing annoying, hard bugs that take ridiculous amounts of time
• Presenting wins from the team at the all hands
• Listening
• Building relationships across teams
• Actually... just like... most things that are useful?

These activities are the heart of the team, and there's no way eng leaders are measuring them all, so these people get pushed down the performance bell curve in favour of more tangible metrics.

Incentives

The moment you start measuring something, you create an incentive to optimize for that measurement. This almost always leads to counterproductive behaviors:

• Measure lines of code? Cool, you get heaps of code. Good luck maintaining it.
• Count closed tickets? Prepare for an influx of tiny, inconsequential fixes.
• Story points? Inflated estimates and sandbagging.

What I don't understand, is that everyone seems to get this. No one I've ever spoken to thinks that this is false. We're so uncomfortable about the idea of not measuring anything though, that we just do it anyway.

Mistrust

Constant measurement erodes the psychological safety that makes high-performing teams, well, high-performing. When engineers feel they're being evaluated based on arbitrary metrics, they're not going to take risks on solutions. admit mistakes, ask for help, or collaborate openly with teammates.

Instead, they focus on gaming the system, where the goal is to hit the numbers, not create value. Engineers start asking themselves, "How can I make these metrics look good?" instead of "How can I solve this problem effectively?"

Trust, but not the type you want

Most of your team are probably friends with each other. This is especially true if you have enforced a Return To Office mandate (I would wager high-control workplaces trend towards both RTO mandates and attempts to measure engineers at a granular level).

Anyway, those friends are just going to collude. Break a PR into 5 micro PRs, add a few extra points, give each other shiny gold stars or whatever your kudos metric is.

In fact, this happens with management too. Most managers also think metrics are silly and they'll happily collude with their teams in order to make them look good.

Diversity

Rigid metric systems often fail to account for the diverse ways people contribute to a team. Women often take on more "non-promotable" tasks (like organizing team events or mentoring) that don't show up in traditional metrics. It's such a ridiculous failure of the system and now women are encouraged to distance themselves from those tasks. Fair enough, but it should never have gone that far.

Individual metrics improve, culture gets worse, team performance decreases. Nice footgun.

This is what you should measure

Here's a different approach: the best metrics might be the ones managers can't see. One of the conclusions I drew from "Measuring and Managing Performance in Organizations," was that effective metrics should be visible to employees but hidden from managers.

Why? Because the moment a metric becomes visible to management, it becomes a target.

"When a measure becomes a target, it ceases to be a good measure." - Goodharts Law

So, give your team every metric you can think of, but also give them a way to prove that you have never looked at it.

The metrics you should look at

There are literally only two things I think you need to focus on to create great teams.

1. Psychological Safety

When people feel safe to take risks, voice opinions, and make mistakes without fear of punishment or humiliation, they make good software.

Google's Project Aristotle found that psychological safety was the most important factor in building successful teams. But that's annoying right? It would have been so much more convenient if the findings were pull requests per day or something.

In short, Psychological safety can be described as so (copied from HBR)

1. If you make a mistake on this team, it is not held against you.
2. Members of this team are able to bring up problems and tough issues.
3. People on this team sometimes accept others for being different.
4. It is safe to take a risk on this team.
5. It isn’t difficult to ask other members of this team for help.
6. No one on this team would deliberately act in a way that undermines my efforts.
7. Working with members of this team, my unique skills and talents are valued and utilized.

Are you 100% confident that you have ways to measure those things? Are you, as a leader, doing everything in your power to advance those metrics?

I'm not. I can do way better on all of these.

2. Ikigai

I have no billion dollar Google project to reference here. But, do I need one? You know this one is true. Do you have an Ikigai Score at your company? If you did, I bet you would notice when it drops, performance drops too.

You're a leader

Your job isn't to create perfect metrics. It's to create an environment where great work can happen.

Org outcomes > Team outcomes

Talk about the issues that the org is having, then look at your team affects those metrics. Stop drilling down metrics to the individual level.

You need brains and heart

You can't just tell your team that soft skills are important but then go promote all the leetcode devs. You know who holds the team together, promote them.

High trust == scary conversations

High trust environments mean people will give harsh feedback about other developers if they're under performing. Adopt a model of radical candor - care personally, challenge directly. Regular, honest feedback is far more valuable than any metric.

Conclusion

Be careful with metrics. They're expensive. But also, if you find a metric that actually works without downsides, start talking! You've hit gold.

What's the best metric you've measured? What's the worst?