Measuring and evaluating Scrum in complex environments

Measure too much in a Scrum environment, and people will focus on data instead of improving their habits. When I visit a company, I always try to measure a couple of variables as a good starting point for a change. It helps people secure some important alignment, make their ideas more mature, and keep goals clear. The metrics also ensure that individuals get more insight into their day-to-day jobs.

Measuring the state of Scrum at the beginning of each engagement is a necessary step. I always clarify with clients that measurements are good, as long as they are an excuse to have good conversations but not to undermine people′s work. I recommend to all Agile professionals and consultants that they collect some initial metrics before jumping into a solution.

Complex environments require different metrics

You can find on the Internet many different approaches to measure Scrum Teams, but most of them are focused on the dynamics and how close to the Scrum framework they are. Making software is a complex process that requires more than just optimizing the Scrum Team. Additionally, some companies are complex environments. As Mel Conway said, "Organizations which design systems . . . are constrained to produce designs which are copies of the communication structures of these organizations." It means that the more complex the enterprise is, the more complicated their products will become. For me, a complex environment includes (among other things):Places where employees do not feel saf

• Companies where there is high pressure and technical debt and/or teams do not take ownership of the code
• Enterprises where there are different definitions of business value
• Teams with a mix of contractors and employees and lack of code excellence
• Organizations with a lot of bureaucracy and/or rules
• Places with a huge number of people working on a product that has no proper vision or road map, or the road map is not realistic or changes frequently

I have been asked several times to help companies improve their software teams. Unfortunately, enhancing just one part of the "system" does not produce a real improvement for the whole enterprise or even the product. If you want to get better at what you do, you need to improve the whole value network.

Optimizing the whole system, not just the Scrum Teams

Just in case you have never heard of this concept, value network refers to all the associated activities that will materialize your service or produce a specific good. The value network process flow starts when a client has a great idea and ends when he gets it realized. In theory, as in practice, you generally want to get feedback after your service is delivered, so that it can extend even longer. I have many theories about why companies don't really understand the concept of optimizing the whole system instead of only the Scrum Teams.

One of the main reasons is the old idea that every enterprise is a software company. It would be easy to assume here that optimizing only the IT department would lead to great outcomes.

The second reason is related to the idea that Agile is only for software teams. For many years, I have seen management pushing ceremonies, practices, values, and metrics onto teams without themselves being able to follow any of them when required. If you focus on measuring and improving only IT teams instead of the whole value network, you will end up refining one area of your enterprise but degrading the others; this is what we call a local optimization.

This obviously creates a much more expensive product that will not solve any real problem.

Measuring the Scrum Teams in complex organizations

In complex companies with complicated products, if you want to create a sustainable ecosystem, it is important to initially consider four actions:Optimize the whole value network instead of improving only your Scrum Teams.

1. Reduce organizational complexity (bureaucracy and structures) across the company.
2. Have the right leadership style in place that constantly reinforces a clear message.
3. Make sure that Scrum operates in a healthy environment with high visibility, clear rules, and high-quality outcomes.

Based on these actions, I have created a way to measure Scrum Teams in complex organizations with complicated products by using the following eight indices:Scrum values and principles in action

1. Use of Scrum artifacts
2. Scrum ceremonies
3. Technical excellence
4. Simplicity in the software codebase
5. Product owner skills and capability
6. Business value delivered
7. Lack of cultural debt and simplicity in processes

Each of these indexes can help you address some specific problems I generally see in companies. These are some of the questions you are likely to get answered after measuring the eight dimensions:

Table 1. Eight indices of the Scrum assessment

Continue reading this article and download for free all the templates needed to measure Scrum in your company

(Spanish) Sigue leyendo este artículo en Espa?ol y baja todas las plantillas necesarias para medir Scrum en tu empresa

You can also learn more in my book: