Three Dimensions of Developer Productivity

This is the latest issue of my newsletter. Each week I cover the latest research and perspectives on developer productivity.

This week I’m sharing A Software Development Productivity Framework by Caitlin Sadowski, Margaret-Anne Storey, and Robert Feldt. This paper was published in 2019 and presents three dimensions for understanding productivity.?

With all the conversation around McKinsey’s new article on measuring productivity, I thought it’d be relevant to share this paper here. And while I agree with the points made by Kent Beck and Gergely Orosz this week in their response to McKinsey’s article, I think this paper offers a simpler (and complementary) path for understanding and measuring productivity.?

My summary of the paper

As we know, developer productivity is challenging to define, describe, and measure, particularly because of the nature of engineering work. In this paper, the authors present a framework for conceptualizing productivity in software development according to three dimensions which they suggest are essential for understanding productivity. They also propose a set of perspectives to consider that help leaders clarify their goals with measurement.

Dimensions of productivity

The dimensions the authors present are:?

• Velocity: How fast work gets done
• Quality: How well work gets done
• Satisfaction: How satisfying the work is

These three dimensions work together and balance each other. The authors argue that any picture of productivity would be incomplete if these three dimensions are not considered.

Velocity refers to the time a task takes, or the time taken to achieve a given quantity of work. The authors warn that “How one may conceptualize or measure velocity is highly task dependent, and the type of task needs to be considered, as well as the granularity, complexity, and routineness of a particular task.”?

Quality encapsulates doing a good job when producing artifacts, such as software. Quality may be an internal consideration in a project (e.g., code quality) or external to a project (e.g., product quality from the perspective of the end users).

Satisfaction is a larger concept that includes feelings such as happiness, autonomy, or flow. This dimension balances the other two: “learning or skill development that may positively impact long-term quality or velocity may manifest as an increase in satisfaction… [Conversely] an increase in velocity may lead to reduced costs, but at the same time it can lead to increased stress for developers (and reduce their satisfaction).”

Note: The lead author of this paper is a researcher from Google; when I recently interviewed two of Google’s engineering productivity researchers, they said they now use the dimensions Speed, Ease, and Quality. “Satisfaction” appears to be one way of measuring the factors within each dimension (e.g., satisfaction with code quality).?

Considerations to help clarify your goals for measuring

In addition to the three dimensions, the authors offer several points to consider in order to clarify the goals for measuring. These include considering which stakeholders are of concern, the level of measurement, and the time period in focus.?

Stakeholders: Different stakeholders (e.g., developer, manager, vice president, etc.) may have varied goals and interpretations of any sort of productivity measurement. “Before trying to understand and measure productivity, it is essential to identify which stakeholders are of concern and what is important to those stakeholders.”?

Level: Productivity may be considered at the individual level, team level, or organizational level. If leaders want to improve productivity at the team level, they should consider the factors that relate to that level, such as coordination and communication. They should also consider how optimizations at different levels may impact each other: for example, interruptions may negatively impact an individual yet lead to a net gain for the team.?

Time period: Productivity goals can vary according to the time period considered. For example, a process change may slow down velocity in the short term but speed up velocity over a longer period of time.?

Going from dimensions to selecting metrics

Going from goals to metrics is not trivial. One technique the authors suggest is the Goals, Signals, Metrics approach. With this approach, leaders start with the goal or dimension, then specify signals that help them know whether they’ve achieved that goal, then define metrics that help them understand whether they’ve achieved the signals.

The authors conclude by providing an example to illustrate how this approach may be used to measure the impact of an intervention:

A manager of a software development team (the stakeholder) in a large software company (the context) would like to improve productivity through the introduction of a new continuous integration system (the stakeholder’s productivity goal). She hopes that productivity will be improved for both individual developers and the team overall (the levels) and intends to measure the change over the time frame of a few months (the time period).

A set of specific questions about productivity improvements arises from considering the productivity goal through the identified lenses along each dimension. Since these questions are specific, it is possible to identify a set of metrics that may help to answer them (shown in the table below).

Final thoughts

In their article, Gergely Orosz and Kent Beck make the point that leaders should focus on measuring outcomes and impact instead of measuring effort (or “activity”) when measuring productivity. This paper complements their argument by providing a set of outcomes leaders may consider measuring: speed, ease, and quality. Additionally, it provides a set of outcomes through which we can measure efforts to improve developer experience.?

What’s your take on this paper and the McKinsey article? Send me a connection request or message here on LinkedIn and let me know.