Agile – An Inconvenient Truth

The Agile movement was supposed to empower programmers. Yet, Martin Fowler, one of the authors of the original Agile Manifesto, has pointed out that the Agile community is now dominated by Agile coaches and other people who have “Agile” in their title or their job role and are not programmers. At an Agile conference in 2018 he said,

“how many people here are software developers by the way?”

A few raised their hands in a room filled with hundreds of people. He then said,

“Mmm a smattering but actually very much a minority. The same problem was that true at the Agile Alliance’s main conference for quite a while that they realized that they were getting a lot of people who were involved in the project management side and things of that kind but not very many people who are the technical people who actually did the work and that's actually quite tragic…”

The question should then be posed, does the Agile community still represent programmers? Or does it now have its own ideas and agenda that are removed from what programmers actually need in order to be effective?

Are “Agile people” going to conferences and celebrating “Agile”, while programmers toil away, frustrated with these methods that are being forced on them?

Slashdot is a widely read programmer and engineer forum. In March 2021 a question was posed in Slashdot, asking, “After 20 years, have we achieved the vision of the Agile Manifesto?” (shown in the image)

Slashdot has a comment and voting process whereby the most upvoted comments are automatically expanded. And guess what?

Among the hundreds of responses to the post – responses by actual programmers and engineers – the top ten most upvoted responses were all negative.

Ten out of ten. (See image)

That is not what success looks like.

The problem is, while there is a lot of debate about the efficacy of agile approaches and the Agile community in online professional forums such as LinkedIn and Slashdot, Agile conferences seem to operate in a different universe, impenetrable to calls for change and disruption. Incremental change is fine, but one may challenge Agile itself at great peril.

It is not that “Agile” is wrong. First of all, “Agile” has become a noun, and it refers to the entire range of ideas, practices, and narratives that predominate within those who use and advocate for “Agile” methods. This is important, because it means that “Agile” is its own thing, not to be confused with “agile” the adjective. The burden of proof is on the Agile community to show that using “Agile” approaches actually lead to agility. It has not done that.

In fact, the most comprehensive study, described in the book Accelerate, finds that some very different behaviors create agility – transformational leadership for one, which is at considerable variance with the self-managing team view that is advocated within the Agile community.

Those who advocate for the Agile community will claim that if you use Agile methods an you do not obtain agility, then one of the following must be true:

1. You are “not doing it right”.
2. You don’t understand Agile.

The community is completely resistant to any suggestion that the core Agile ideas are incomplete or flawed in any way. And as my editor at Pearson Publishing once put it, “The Agile community is highly insular”. For example, while the community realized in the mid-2000s that Agile ideas of the time failed to account for an organization’s culture, to this day the Agile community at large has a very low literacy with regard to organizational culture theory. Yet organizational culture is central to the challenge of how to make an organization more agile, in a true sense. That’s the insularity my editor was talking about.

The same lack of knowledge and awareness is also true with regard to business theory, management theory, leadership theory, finance, operations research, cognitive science, and behavioral psychology. Yet incorporating all those areas of thought is key to understanding agility.

One area which the Agile community has sought to include is product design. It was a latecomer, but better late than never.

The reality is this:

Agility cannot – will not – be achieved by having people follow a set of “Agile” practices, or an “Agile” framework or methodology such as Scrum, SAFe, or any other. It will not. That is certain.

It is not that these practices or frameworks are bad or wrong. It is that starting with those is like copying a set of instructions to paint a picture.

The result will be pedestrian.

You won’t be competitive.

Agility is behavioral. We at Agile 2 Academy know, because we have studied highly agile companies. Those companies do not share any practices or processes. What they share is certain behaviors. One of the behaviors is that managers expect that people will try things before being sure. Another is that managers expect people to solve problems – not stay in their lane or complete their “story”.

When I say that these and other behaviors are expected, I don’t mean that they are like values on a poster. I mean that if people don’t behave in these ways, they get in trouble. Not behaving this way is seen as improper, like being habitually late for important meetings, or not completing work, or being rude to colleagues. It is strongly frowned on and results in clearly evident disapproval from others and poor performance reviews.

These and other behaviors are what we saw in common across all of the highly agile companies that we studied. When I say that these companies are agile, I mean that they have demonstrated actual business agility: the ability to rapidly and efficaciously change direction at large scale.

Agile’s inconvenient truth is that it does not work.

Again, it is not that the core ideas are wrong. It is that,

1. The core ideas are expressed as unactionable maxims, without explanation or context. As a result, they resonate with people who don’t need the advice, and read like a set of rules to those who do need the advice.
2. Many core ideas are correct but miss the actual key issue. For example, while self organizing teams are a sign of a healthy culture, they are actually a poor starting point. What new teams need is good leadership.
3. The core ideas are incomplete: for example, they ignore the role of leadership, product design, and data, and these omissions have led to misunderstanding about what is important.
4. The frameworks being sold as “Agile” are not actually agile. They are practice first and behavior second, and so they solve the wrong problem. In fact, people should pick and design their own practices, informed by others’ experiences.
5. The Agile community is insular and dogmatic, and is deeply intertwined with those who sell the frameworks.
6. The Agile community, by and large, does not understand the true nature of agility in an organization. Of course, there are many exceptions – people who read beyond the Agile dogma.

So while many Agile ideas are correct, they are actually poor advice, or incomplete advice, and the community that supports them does not have the acumen to have correct discussions about true organizational agility.

Ironically, if you listen to talks by the authors of the original Agile Manifesto, some of them “get it” with regard to agility, and are very good at articulating what matters. For example, Martin Fowler clearly understands what matters, and is very good at explaining it. I frankly prefer the writings of Martin Fowler over the Manifesto itself.

And a key sign of a thought leader is, Are they able to criticize what they have created? Which of the Manifesto’s original authors are willing to call attention to what has gone wrong? A few have, but not many.

We need to learn from the Agile experience but look beyond “Agile”. We need to look at behavioral psychology, leadership theory, and the other disciplines that I mentioned earlier. And we need to unshackle ourselves from the entrenched thought patterns that have ossified thinking in the Agile community. We need to think out of the Agile box, and be inquisitive about how to achieve true agility.