"Yes, you can measure software developer productivity." Really?

If you have even a passing interest in Engineering Management topics, you've likely not missed the recent controversy over McKinsey's article on measuring software developer productivity [1]. In essence, the consulting giant claims to have found the holy grail — a universal, quantifiable method for measuring developer productivity.

The most popular counter-argument comes from Gregerly Orosz and Kent Beck in their two-part article on The Pragmatic Engineer [2,3]. While they started on a note that the McKinsey article might be “so absurd and naive that it makes no sense to critique it in detail”, they proceed to do exactly that. Which brings about a great and detailed article, but also distracts from the essence of the most fundamental issues with the McKinsey report.

So, why is this McKinsey article so "absurd and naive"?

In my opinion, there are three fundamental flaws (next to the many smaller ones). Alone, these three points dismantle their primary assertion: that they've developed a serious, novel method for assessing developer productivity.

• The McKinsey article largely reiterates well-known concepts and re-combines them in an unsurprising way in a "new" framework. Everything presented appears to be a rehash of ideas I've encountered in various organizations years ago. They might argue that there's a secret sauce or proprietary algorithm they're not disclosing. But the only visible difference is the passionate pitch they wrap it in — culminating in the time-tested consulting argument along the lines of "trust me, this worked well at other big industry players". It's a gigantic smokescreen, and misery is lurking behind it.
• The proposal lacks clarity and internal consistency. I'll name just three examples. The authors mix general best practices to enhance productivity with the discussion on how to best measure it. They appear to confuse actual productivity (which they do not define) with associated metrics. In section “Avoiding metrics missteps”, they warn about two main mistakes. The first can be read as “do not use oversimplified metrics”. Ironically, they propose doing exactly that in several places. The second "mistake" on their list, strangely enough, could be paraphrased as “face the reality that simplified metrics don't work”. Clearly, the authors are not guilty of making that "mistake".
• They’re shockingly naive with regards to how engineers, and engineering organizations, respond to such frameworks. Here’s the thing: they propose to calculate scores, KPIs. As far as software development productivity is concerned, these are, at best, proxy metrics. They may provide some guidance but are hardly an exact measurement. And they can easily be played with. Now, capable engineers are sharp analytical thinkers who enjoy building, reverse-engineering, optimizing, and playing with complex systems. Guess what's going to happen?

I rest my case.

Measurements are valuable. But KPIs won’t give you an accurate number on developer productivity, most certainly not where innovation is paramount.

If you genuinely value developer productivity, the formula is simple: Hire a talented team, keep them motivated, clarify your business needs, and eliminate any roadblocks. Carefully use a few proxy KPIs to monitor progress and team health, but don't conflate these with developer productivity. And for heaven's sake, don't demean yourself by using a KPI derived from such a framework as an individual's performance rating.

On a positive note, most of the McKinsey article isn't really about measuring performance. Instead, it proposes using well-tested industry best practices for enhancing productivity. Surveys, for instance.