Failing Agile: Agile is corroding and decaying before us, and it is entirely our collective fault.

Across social media platforms, Agile Manifesto authors and luminaries from throughout the Agile industrial complex are complaining of the misdirected, misattributed, ineffectual, and misshapen form into which “Agile” is devolving. From lamenting historical revisionism on the part of Alistair Cockburn and Ken Schwaber (with commentary from Ron Jeffries and others)[1], to the full-throated admonishment of “snake-oil” consultancies and coaches “pushing” insufficient and ineffective solutions on the part of Tobias Mayer and others[2], the creators and early advocates of Agile seem now to be locked in a Frankensteinian war against their own creation.

In fact, Scrum Alliance, “the only member-driven nonprofit certifying body in the agile space…”[3] recently won a legal injunction against Scrum, Inc., the company created and owned by Jeff Sutherland, co-creator of the Scrum Framework, over trademark infringement relating to specific Scrum-related terms. Just to be explicitly clear, that is a case of a “non-profit,” agile certification organization winning a lawsuit against one of the creators of the framework for which it provides certifications. 

Yes, makes my head spin, too.

For a philosophy which promotes “individuals and interactions” over “processes and tools,” and the Scrum framework which avows courage, focus, openness, respect, and commitment, in the view of this author, maybe now would be a good time for everyone to take a deep breath and reflect for a moment or two on just what exactly is occurring out in Agile-land.

What’s occurring (or has occurred, to use the current tense) is Commodification, Commoditization, and Late Majority adoption.

Background

The term “Agile,” as most of us know, was branded (? ? ?) into existence at a fateful meeting of futuristic software development thinkers in 2001. Coming from different backgrounds and representing different approaches to development frameworks and methodologies, they came together to explore and reveal what it was about their approaches that unified them. 

Out of their exploration and work came the Agile Manifesto for Software Development, a set of four values and twelve principles which defined what it meant to “be agile.” The authors, having created their Frankenstein, returned to their individual disciplines to continue building their respective empires, but under the auspices of their new creation, the Agile Manifesto.

   Obviously, I was not personally present at those meetings. However, over the years, I’ve been fortunate to have met several of the authors of the Agile Manifesto in person, and I read and follow their work in books, on social media, and via podcasts, and their appearances at conferences. I feel fairly confident in making the assertion that none of the Manifesto authors went into the 2001 retreat at Snowbird, Utah with the express intention of building business consulting empires based on Agile certifications.

Having lived and labored through the failures of emerging software development practices throughout the 1980s and 1990s, these thought leaders, engineers, and luminaries were simply practicing what many managers and leaders preach: they were passionate about their craft and about improving their business and industry, and they were actively attempting to do something about it.

   Fundamentally, they simply wanted to help companies, teams, and individuals grow, progress, and improve in their ability to build and deliver software effectively.

   Yet, as every creator (indeed, every parent) knows, once you bring life to a creation, that creation typically begins to formulate its own ideas, its own thoughts, and eventually pursues a life all its own. Just ask the good Doctor Frankenstein.

Commodification and Commoditization

   As the Agile movement grew, and as companies began to adopt, adapt, and talk about it, the inevitable happened: Agile transformed from being an idea, a concept, a movement, a set of values and principles, and a way to help organizations, to becoming a product in itself. It changed from being a way to achieve a goal, to being the goal to achieve. 

Agile evolved into something which could be sold by consultants and coaches, trainers and staffing companies. The introduction of certification schemes to ensure that people were “qualified” (after a whopping two-day course) to lead agile change management initiatives, agile teams, and help struggling organizations realize the value and power that agile methodologies could provide, began to emerge and thrive.

   An entire industry in itself was born from nothing more than providing certification training in various frameworks and methodologies relating to Agile. So-called “nonprofit” certification companies were established to measure, monitor, and sell the dream of agility. Frankenstein’s monster had indeed broken free from the castle and was roaming openly across the land.

Agile at this point had undergone commodification. For those unfamiliar, commodification is the transformation of a previously untraded good or service (such as ideas, nature, information, or people) into a commodity, or object of trade. For example, your web browsing history and trends in the early 1990s was nothing more than information stored locally on your computer. Today, it has been transformed and commodified by companies like Google and Facebook to be valuable information which they can sell to potential advertisers for significant sums of money.  

What triggered this shift in the environment to move Agile from a philosophy to a commodity? In many ways, the catalysts for Agile to transform and explode the way in which it did could not have been more perfectly prepared and aligned.

Many tech companies were struggling as the burst of the dot-com bubble wreaked havoc in the industry during the early 2000s, while technological advancements were driving rapid change and presenting immense opportunities across the business world for those companies who could capitalize on them. Combined with the rather negative history of software development project management practices throughout the 1980s and 1990s, the promise of agile frameworks and methodologies which held that software could indeed be developed better, faster, and cheaper, couldn’t have come at a more opportune time. 

Arising from a managerial and corporate malaise in which many companies and industries were rethinking fundamental issues about how to operate, lead, and innovate in the emerging technology century, Agile initially offered a significant competitive advantage for those who could employ its techniques sufficiently and extract the rewards which it promised.

When companies looked for support in entering this brave new frontier, Agile certification companies and agile consultancies were ready to answer the call. This new industry arose from the hard work of those early, visionary, Manifesto authors, but grew so quickly that it exploded well beyond the bounds of the original vision. Instead of a vehicle for service and support, it became a vehicle for excessive expense, pointless and unnecessary experimentation, and greater largess and ineptitude.

As the Agile movement transitioned into becoming its own product, and as industries caught up with the lingo and jargon, cracks began to appear. The most lucrative of which could best be summarized as “how to do agile in large enterprises?” After all, most of the agile case studies and instructional material which had been put into training in Agile’s early days addressed topics such as job roles and functions within agile teams, process specifics for individual task accomplishment within team boundaries, and meeting cadences with specific, prescribed inputs and outputs in order to conform to whatever specific methodology or framework (typically Scrum) was being followed. 

In other words, Agile was extremely tactical in its implementation. This was a natural result of two-day training courses for certification. Trading speed for intellectual depth, which at the time likely made sense due to the newly emerging nature of Agile as a commodity, most frameworks and methodologies left the machinations of strategy and scale - by which we mean coordinating the efforts of hundreds of teams across a large company - to the company itself to solve. As companies struggled to devise ways to implement Agile principles and process across an entire enterprise, smart consultants saw the opportunity for yet more products.

Enter “scaling frameworks,” another aspect of the agile product realm which could quickly be commodified and sold to an increasingly hungry market. To answer the question of scale, consultants and consortiums came up with “scaling frameworks,” such as Large Scale Scrum (LeSS), Scrum@Scale?, Nexus?, and the Scaled Agile Framework? (SAFe). Naturally, the opportunity of providing an enterprise-level solution for a consulting company brought with it an enterprise-level price tag, making it a lucrative business endeavor well worth pursuing.

Naturally, all this was also a boon for the training and certification sector within the Agile industry. Scaling meant additional certification levels, additional consulting requirements, and additional sources of revenue. The model was actually far better than any pyramid scheme has ever hoped to be, because it cannot empirically be proven to be ineffective (at least, not yet).

Let’s examine a common scenario: a struggling company is sold on the idea that they need to become agile, and in order to do so will need to invest in process adoption and workforce training. Fortunately, there is a simple roadmap for training and certifying the entire organization in any chosen Agile discipline. 

Not only is the cost of certification significant, but many agile certification bodies require payment of annual dues in order to maintain certifications as “current,” creating ongoing revenue streams (regardless of the actual value such “currency” actually provides to the organization or the certificate-holders).

As additional frameworks and methodologies continued to enter the market, and tangential or corollary products (business agility, DevOps, agile coaching, etc.) began to flood in, Agile further became commoditized. 

Commoditization is typically viewed as positive, whereby an increasingly undifferentiated offering of similar (possibly indistinguishable) products (Agile frameworks, tools, and certifications in our case) results in lower prices for consumers of those commodities. Product differentiation from the early days of the product gives way to simple price competition as products meld, mesh, and mature in the competitive environment.

However, in terms of a product which is based on a system of values and principles, and which is intended to be used to guide organizations in their own development and delivery of value, through which they derive revenue, both commodification and commoditization can (and did) unleash a torrent of low quality, misinformed, misguided, and misdirected competitive products which are fundamentally incapable of achieving the vision or purpose of the original.

This is one reason why the authors of the Agile Manifesto, and thought leaders throughout the industry, are lamenting the current state to which Agile has devolved.

Yet surely all is not lost? Is it too late to soothe and coax the monster back into the castle? I would assert that yes, indeed it is, and that is due to the late stage of product adoption in which Agile has now progressed.

The Late Majority (& Laggards)

Everett Rogers in his 1962 book Diffusion of Innovations (The Free Press: New York, 1962) generalized the bell curve through which societies or groups adopt innovative ideas or technologies. For anyone unfamiliar, the curve begins with Innovators and Early Adopters before continuing through the Early Majority, Late Majority, and Laggards. In short, Innovators adopt new ideas or technology to experiment and try to find ways to gain a competitive advantage.

By the time an idea or technology has reached the Late Majority, it has effectively become a CODB in business terms (Cost Of Doing Business). At that point in the adoption curve, those who do not have access to or do not use the technology in question are disadvantaged merely by the fact that they are losing out in some way to those who do.

Agile frameworks and methodologies today have progressed, at the very least, well into the Late Majority phase of adoption (and, more likely, to Laggards). Gone (or very nearly gone) are the exhortations of “that agile stuff doesn’t work!” Instead, companies and industries are well aware that if they are not “agile,” they are losing ground to those companies, startups or enterprises, who are, somehow.

Hence we Agilists find ourselves today in the unenviable position of being stuck in an increasingly commoditized market racing toward (or through) the Laggard phase of idea and technology adoption. The industry is caught in a “race to the bottom,” where companies want the cheapest price, the product itself is undifferentiated, and the demand signal for expert help is beginning to fade simply due to market saturation. This places increasing pressure on those consultancies and companies who have metaphorically put all their eggs into the Agile basket and have no other product or service apart from Agile training, certifications, and consulting. 

That pressure means they will increasingly agree to compromise on their values and principles in order to acquire and maintain revenue. Even those companies and individuals who advocate against senseless training and hollow certifications will agree to provide training and certification, as the alternative is insolvency. Nevertheless, this is fundamentally a Faustian bargain.

To be sure, the Late Adopters and Laggards guarantee no shortage of work, but with consultancies, staffing agencies, and market saturation driving prices to the bottom, there increasingly seems to be a light at the end of the tunnel, and it is likely the proverbial train. 

Where to go from here?

“I don’t care if it’s lean or agile, there’s no silver bullet where if you just follow this formula that somebody else followed, you’re going to be great...” [5]

-Mary Popendieck, author of Lean Software Development (Addison Wesley: 2003).

To come full circle, the truth is that the “Frankenstein’s monster” of Agile is a beautiful creature. The origins and intent behind the Agile Manifesto are noble and worthy of pursuit. The authors of the Manifesto sought to improve and grow their industry - and in many ways they have achieved that. Yet sometimes the price of success is the sacrifice of the soul, as certainly could be the case with Agile.

In my personal view, Agile is but another step in our collective evolution as a species. From fiefdoms and guilds to Taylorism and management science, to organizational design, and, yes, into Agile, we seem dedicated to “uncovering better ways of working, by doing it, and helping others do it,” to riff from the opening of the Manifesto itself.

This is the inflection point at which we find ourselves today. Agile was intended to improve software development, but increasingly it is sought as a means for improving the way organizations and companies function, writ large. Yet Agile was never intended to fulfill such a mission nor span such voids.

Instead, what is needed for such tasks are multi-disciplined approaches which leverage our collective wisdom across diverse but related domains, such as business and management, finance, sociology, neuroscience, industrial-organizational psychology, complexity science, organizational behavior, professional coaching, learning & development, and, yes, software engineering and computer science, as well as others.

Technology has evolved and advanced. We collectively know so much more about building and delivering software today as compared to 2001, or 1990. The tools, techniques, and training available are worlds beyond where they were even a decade ago. As times and technologies have changed, organizations and yes, Agile, have changed as well.

Through commodification and the spread of adoption, Agile isn’t what it was, and may never be what it was intended, but perhaps that’s the lesson we must collectively learn. We have taken a great, collective step in the right direction. Now we should embrace what Agile teaches: that we can adapt and apply what we have learned toward what needs to be done next. Forget the problem we set out to solve, and instead look to the problems emerging before us. Problems which require a new approach, and a new paradigm.

Our monster doesn’t need to be killed, but it does need to evolve. Maybe the time has come for a new manifesto.

I'm Chris Alexander, the Director of Agile Delivery at EXPANSIA Group, and an Enterprise Agile Coach (whatever that means). I'm also a student of complexity theory, industrial-organizational psychology, organizational behavior, and, yes, agile. I enjoy counterfactuals, paradoxes, and non sequitur, but only ipso facto, in media res, and de rerum natura.

1. https://twitter.com/TotherAlistair/status/1294663110196891648 (accessed Aug 19, 2020).
2. https://www.dhirubhai.net/posts/tobiasgmayer_if-you-buy-snake-oil-from-a-salesman-because-activity-6701581770179059712-deaI (accessed Aug 19, 2020).
3.  https://www.scrumalliance.org/about-us (accessed Aug 19, 2020).
4. Scrum All., Inc. v. Scrum, Inc., Civil Action No. 4:20-CV-00227, 33 (E.D. Tex. Jul. 16, 2020), https://casetext.com/case/scrum-all-inc-v-scrum-inc (accessed Aug 19, 2020).
5. https://builtin.com/software-engineering-perspectives/lean-agile-methodology-software-engineering (accessed Aug 19, 2020).