Maturity Models: How Useful Are They?

If you hire a consulting company to help to improve your IT agility, the first thing they will propose is that they conduct a “maturity assessment”.

An Agile maturity assessment is a big win for a consultancy, because it is a largely academic exercise, and so it is easy to do. The result is a report, which scores your organization on a chart (usually accompanied with a proposal for Agile transformation assistance).

For each category of the chart, your teams are rated from a range of beginner to advanced, depending on what “practices” the teams are using and what “attitudes” the teams exhibit. Here is one such model from Thoughtworks.

One problem with an Agile maturity model is that it presents a prescriptive idea of what “ideal” is. Yet, Agile often looks different in different organizations, and for good reason.

For example, a company that builds embedded software that controls expensive machines cannot test quickly and often in the same way that one could for a server based Web application. Actually, they could, but it would require a very large investment in automation to achieve that; the cost/benefit tradeoffs need to inform the decision. Blindly scoring someone on a maturity model is a simple-minded approach.

 “Any fool can know. The point is to understand.” - Albert Einstein

Whichever maturity model one uses (there are many competing ones), it is important to not be too detailed or prescriptive, but instead focus on higher level concepts. For example, while unit testing was considered a crucial element of Agile years ago, today the focus has shifted to automated acceptance testing, as systems have become larger and more complex and integration testing has become more crucial than unit level testing.

Either way, having frequent automated tests are crucial and are a sign of “maturity”, but specifying that they should be mostly unit level or integration level is something that should be decided on a case by case basis—not something that is scored.

This points to one of the things that is often left out of maturity models: integration. The focus is often on team level practices, but in my experience, the stumbling blocks that IT organizations tend to face today are the cross-team ones.

The stumbling blocks that IT organizations tend to face today are not at the team level, but at the cross team level.

Focus Effort Where it Makes the Most Difference

An important thing to realize is that if one were to do all the things that are recommended for building and releasing software, one would probably go out of business. Software assurance practices are like insurance: you have to decide how much you need, based on your risks, and hope there is still enough left over to fund investment.

This is why one should focus on the problems that your organization is actually having, and look at which practices might alleviate those. If you try to implement every practice in a maturity model, you will spend a-lot of unnecessary effort, and you will extend the time to when you start to receive tangible benefits. This is a Theory of Constraints approach: determining ways in which your teams are being held back, and instituting practices that overcome those.

Some teams perform really well by using only some Agile practices—not necessarily all. It depends on the team, their environment, their customer market, and what they are building.

Forget “Maturity”—Decide How You Want to Work

Another problem with maturity models is that the term “maturity” is a bad metaphor. Humans mature through stages. However, teams do not need to. A team—or a collection of teams—can often leapfrog stages of “maturity”.

For example, referencing the Thoughtworks maturity model, if a team is using manual testing, it can decide to start using automated testing for all new work, and progress straight to the level of fully automated regression testing—at least for new functionality. All they need to do is discuss how to do that, and start doing it, and gradually refine their approach. They do not need to “take baby steps” or go through interim levels of maturity.

Indeed, I have found that teams can adopt a surprising number of new practices in short order, if you coach them through considering those and the benefits.

A Model Is Not a Plan

Maturity models are useful for considering ways to improve, but they are models. They are not effective as a scorecard, because teams can often perform really well even if they are low in “maturity” in some areas. A maturity model is also not a good “plan” for what to improve: you should look at what is holding people back, and create a unique plan for the situation.

It is also a waste of time to assess maturity for all of your teams. A more economical, faster, and effective approach is to find out which are the best performing and worst performing teams, check in on those to observe how they work, look at some of the code and the tests, look at many of the various build jobs, look at quality metrics and incident root causes, and form a picture of how the teams are working together as a system. Assessing the “maturity” of each and every team is not necessary, and will cause you to lose sight of the root causes and what is really constraining your organization’s performance.