Agile – I Told You So

I think it is no longer controversial that the Agile movement is in decline. When I made a post about it a year ago, the post went viral, receiving a million views in only a few days. But since then, not only have prominent Agilists like Stefan Wolpers come out and shared the view that the movement is in decline, but the Agile Alliance itself has been acquired by the Project Management Institute (PMI) – long considered to be “the enemy” by many if not most of those who write about, teach, and practice Agile methods.

So the Agile movement is in decline. That is no longer in question. Not dead, but the Agile ship is sinking.

There are many factors, but one of them is an uncomfortable truth: a lot of what we consider to be “Agile” is not right. I am sorry, but that is the truth; and I have been saying that for a long time, with articles such as,

The Agile Community Embraces an Unworkable Fantasy (won an award)

Why Self-Organizing Teams Don’t Work (40,000 reads in the first week)

Agile – An Inconvenient Truth

Agile Core Ideas – Something Is Not Quite Right

Scrum Was Unethical From the Start

Why “Agile” Does Not Produce Agility – and What Does

Agility Without “Agile”

and countless more (over 100 actually).

So I am here to tell you, I told you so. And by the way, I co-founded and was CTO of a very successful software company that built mission-critical things very quickly at scale. So I am speaking from experience – not Agile theory.

Agile is not dead though – not yet. The movement has a lot of momentum: Scrum or SAFe roles are in use in the majority of IT organizations, and even in a large percent of engineering organizations, as well as other kinds.

But the ship is sinking. Here is what to do, individually and collectively – unless we want it to sink completely and be lost to the history of bygone IT fads. (Remember Lean Six-Sigma?) I think that would be a shame, because the spirit of Agile was sound, and we don't want to lose that.

We Created Agile 2 to Save Agile

Some people have jumped to the incorrect conclusion that I am “pushing some framework” as an alternative to Scrum or other purported Agile frameworks. And yes, I assembled the team that created Agile 2, but I rarely mention Agile 2. I did not create Agile 2 – the Agile 2 team of fifteen people did.

As it turns out, most of my present company’s work is based on foundational things like leadership, group behavior, cognition, and operations research. We don’t use frameworks and we don’t use Agile 2.

Agile 2 was created to call attention to the things that matter. It was not designed as a guide to follow. It is a set of reminders – “don’t forget about this! And that!”

We felt it was needed, because the Agile community was not talking about those things, but should have been. And the Agile community was amplifying a lot of extreme ideas. Extremes usually don’t work, except in extreme circumstances.

The Agile community had quickly – from early on – become an echo chamber, in which virtue signaling was based on how extreme or novel an idea was. My editor at Pearson Publishing described the Agile community as “insular”. Agile narratives are rife with claims that are not validated by research. Agile had become a cult sustained by framework certifications, and that was a house of cards – eventually, the corporate world would figure that out.

With Agile 2, we wanted to restore rationality and calm, and focus on what actually generates true agility. Agile 2, which was developed over a period of six months, is thoughtful and measured. It shuns extremes or wild claims. It embraces the ample research that exists in areas such as leadership, behavior, and cognition.

But now we are where we are. We did not save Agile: Agile 2 was not enough to steer the big ship of the Agile movement out from among the reefs of frameworks and extremes. It ran aground anyway, and now it is sinking – fast. Can it be saved?

Agility Is Behavioral – Frameworks Are Irrelevant

Here’s what killed the movement: the frameworks. That’s my opinion, and it was the opinion of most of the 700 respondents to my viral post.

Agility is mostly behavioral. I know that because we (my current company) looked at five companies that were highly agile in a true sense, at scale. They were agile in the sense that they were able to pivot strategy in a big way, quickly and successfully. The companies were SpaceX, Netflix, Amazon, Google, and Spotify.

What we found was that Agile frameworks were largely on the periphery in those companies. Instead, we found that each of those companies generated its own approach, sometimes using frameworks for ideas, but in the end creating its own methods. One of them, SpaceX, doesn’t even use the word “Agile”, and yet it is arguably the most agile of all, in a true sense. (We interviewed people who worked there.)

We also found that there were some common leadership traits in play. Some of them were,

• The expectation that people will solve problems.
• The expectation that people will not wait.
• The expectation that people will go out of their lane.
• The expectation that people will talk to whoever they need to – not ask permission.
• The expectation that people will prudently try things before being completely sure it will work.
• The expectation that people will think holistically, rather than only on “their piece”.

Leaders set expectations and make it clear how they want people to think and act. That is crucial. In these companies, the expectations generated a culture that was highly agile.

The leaders created the culture through their expectations.

Agility arises through the expectations of leaders. Frameworks like Scrum and SAFe have nothing to do with agility. You can debate all day if the frameworks are good or bad, but agility is something apart from the Agile frameworks – a great irony.

The irony on top of the irony is that agility – true agility – is more important than ever.

What Matters Today

This means that the Agile community needs to shift its focus from these frameworks, to focus on what actually generates agility.

The community also needs to show humility. It got some things wrong – a lot of things. The frameworks were a bad direction; the absence of any mention of leadership in the Agile Manifesto is a gaping hole. The dogma that characterized the movement for 20 years was very toxic. Finally, the blind acceptance of unproven ideas not backed by any research has given the movement the character of a pop culture – to reapply the comparison famously made by Alan Kay.

The Agile community needs to own up to these dysfunctions and discuss them, to make it clear that the community is ready to shift. That is the only way to regain the credibility that the Agile community has lost.

A lot of companies are now replacing Scrum roles like Scrum Master and Product Owner with more traditional sounding roles such as delivery lead and product manager. That’s a good thing.

The Scrum Master role has very weak accountability for product delivery, whereas a delivery lead is usually explicitly accountable for getting the work done. Similarly, while the Scrum Product Owner is, in theory, accountable for the product, in reality it is usually a business analyst, because there are usually many teams and the true accountable person oversees all of the team level Product Owners.

The Scrum roles just don’t make sense in the real world. They never did. Scrum is and always was a made-up method not backed by any research or validated theory. (Having studied control theory, I can tell you that Scrum’s claim of being based on control theory is nonsense; and by the way, here is another method being pushed by the same person: https://www.frequencyfoundation.com/about-us/ – be your own judge.)

One thing the Agile community got really wrong was that it promoted some false equivalences: it equated accountability and management with “punishment” and “pointy haired boss”, respectively. But accountability is not about punishment: it is about coordination and communication. Accountability identifies who you talk to about an issue. And management is essential for any organization: good managers are good leaders. Effective management includes leadership – at least according to the definition of “manage” that is taught in business school. Leadership is not separate from management – it is part of it.

The dilemma is, and has long been, that most managers are not good leaders. That is not a surprise because few have had leadership training. Leadership is a complex subject, and it is something that has cognitive and behavioral aspects. Some people are natural leaders in some ways, but rarely in all ways; and teams need many forms of leadership.

The whole issue of leadership is multifaceted, and yet it is something that few people understand well. To have good discussions about leadership, and to have good leaders, we need people to learn about it. Learning is the beginning of becoming effective.

So instead of wasting time learning Agile frameworks, people should be learning about leadership. That is the path to agility, because it will make them more effective leaders; and as we learned by studying five highly agile companies, their agility all stems from the cultures created by their leaders.

We (Agile 2 Academy) have a leadership program designed to cover the range of issues that tend to arise in a product development setting. It is based on established research in leadership, behavior, cognition, and operations research, and we highly recommend it. But there are other sources as well.

Embark on a learning journey. Become a better leader. It will benefit you for the rest of your professional career – and your life. That is what the entire Agile community needs to do: read outside of the Agile echo chamber: the answers to what generates agility are there. Start talking about true agility, based on what actual research tells us, rather than frameworks and made-up extreme ideas.

If you want to join our community that is interested in the intersections between leadership, agility, and innovation, join it here: https://agile2academy.sutra.co