Agile coaching: a black hole moment or big-bang evolution?

The year of 2023 has not been very pleasant for the software services industry. With geo-political events, high inflation, fear of recession pro-longing lot more than expected and resiliency efforts, tech firms like McKinsey, Meta, Microsoft, Google, Amazon (and many more) trimmed down their workforce significantly. One of the themes since last 1 year has also been reduction in the agile staff among other tech roles - agile coaches to be specific.

The news of Capital One laying off 1100 agile roles earlier in year 2023 sent chills in the agile industry. This, coming from a company that had published stories of agile and scaling success in preceding years was even more shocking. What happened here? Similar news was heard from other tech companies as well. What's causing this downsizing? Have they achieved the desired agile outcomes? Do they not want to be agile anymore? Or were they simply overstaffed?

A little flashback

When the agile manifesto was published in 2001 (although Scrum/XP existed for years before this), there was no such role of an "agile coach" at that time. Agile Manifesto itself mentions "we are discovering better ways of ....." - where "we" meant the entire team together is discovering better ways of working. However, experienced people from that era took on the responsibility of keeping the team sane while practicing and bringing agile practices to their teams. These ambassadors had seen what the development looked like without agile practices, and could really go deep into what it takes to deliver high quality software with hands on experience.

Then came the mid-joiners, with more demand picking in the industry of such people who could help the team adapt agile practices. With Scrum becoming more popular, Scrum Master role was sought after even more. With increasing demand, the role morphed into Agile Coaches who could coach larger teams and brought know-how of various agile frameworks with assessment and suitability.

Then came the Laggards - who joined at the peak of agile movement. There were companies that felt left out if not practicing agile (FOMO). Impressed with inter-galactic benefits of agile sold by consulting firms, many of them also started "blanket" agile transformation programs covering 10s of teams in parallel. Needless to say, to cover such large programs, you also need as many Scrum Masters, Agile Coaches, Tribe Coaches, Leadership Coaches .... (phew!!). Departments and CoEs for agile sprung up accompanied by recruitment drives to fulfill such roles.

So what went wrong? Upon analysis, you will realize that it's a black hole moment - a shining star collapsing due to it's own weight and gravitational force.

What factors are contributing to the black hole effect?

A black hole is created from a dying star when it has burnt out all it's energy and the internal gravitational pull is so strong that it tends to pull itself in. Has the agile industry burnt it's aura and dying of it's own weight?

From my own industry experience, there are several factors resulting in recent reductions of agile coaches we are seeing for this skill set:

1. FOMO (Fear Of Missing Out): As mentioned above, many companies started practicing agile because it just made sense, and many others were doing it. However, they didn't clearly define what problem they wanted agile to solve for them - even if the intentions were right. In one of the consulting assignments from a previous employer, I was asked to provide 20 Agile Coaches (and not asked to bring agility) by a large scale international bank to stand up 20 Scrum Teams so that the agile transformation targets for that year could be met! Imagine the weight and gravitational force of justifying the outcomes of such a program next year with that much of staffing!
2. Certifications industry: I am personally not against any certifications (with myself holding many of them) as they provide credentials on your knowledge level of a particular field/framework/technology etc. The sad part in the agile industry has been that they were "printing" agile coaches after attending a 2-day class and a 40 minute online exam with no real way to gauge abilities to drive agility. Many people simply re-branded themselves after attaining these certifications without any real prior experience of coaching any teams. This includes professionals who have never been part of any real delivery projects. Worst, the interviewers found it hard to gauge these gaps as well. With such a skill gap, driving real outcomes is highly questionable.
3. It's all business after all: There's nothing wrong with selling something that's in demand by the clients. With more and more companies forecasting large agile transformation initiatives resulted in consulting firms selling "agile transformation on a platter". This included initial trainings, setting up co-sourced pilot teams to deliver small parts of projects to demonstrate agile practices and org structure, followed by longer term deployment of few coaches. The missing part though, both from client and consulting firm in such engagements was sustenance of these practices, let alone changing the mindset of people. Once the consulting firm reduced/removed their consultants, most organizations went back to their older ways of working. It is evident, that in such case, it is easier to claim "agile didn't work for us".
4. Coaches not demonstrating agile value proposition: As mentioned above, many professionals became agile coaches because it was in demand, and they wanted to change their career for good reasons. Influenced heavily by certifications (and their versions), many of them focused on maturing teams more on "practices" and lesser on "outcomes" and "mindset". Also, I have come across many coaches who feel "job secure" by staying with one team for years, rather than removing the team's blockers and moving on to the next team. Irrespective of the duration of agile coach engagement, it is critical to continuously demonstrate value of coaching with tangible business outcomes, in the absence of which the efforts of coaching get diluted. The problem is multiplied if the expectations and goals of agility are not clearly defined by leadership either. Many leaders therefore started re-thinking (rather challenging) the value addition of agile coaches.
5. Companies that have been there, done that: I have come across promising examples of several programs and organizations where the agile transformation movement has really shined like a star! Executing any project utilizing agile practices is a de-facto of their life. Agile planning and execution, cross-functional teams, product evolution, customer feedback, continuous delivery etc. are a default practice (yes there are such places!). Such practices have also matured to larger group of teams and departments. After reaching such a state of maturity, the question is - what value addition is there from Coaching capability? Yes, the sports team argument is always true (because world's best football teams still need coaches), but with time it becomes harder to justify the same strength of coaching team.
6. Shrinking interest in agile as a subject: I have always valued going to agile conferences learning from industry practitioners as well as sharing my own experiences as speaker/moderator with halls full of participants. This was a common site up until COVID started, post which almost every conference went virtual. But so did the knowledge resources! By this time, agile as a subject not being that new anymore, democratization of knowledge and case studies available for few bucks on online learning platforms (or free on YouTube) resulted in lesser footfall in conferences that opened up post COVID. My last in-person conference had less than 40% of expected attendance, same feedback from other conferences. Fewer than ever people are willing to pay and attend a conference to learn when they can do it on-demand online. Also, as mentioned above, most organizations are not that new to the subject either with many successful cases.

As an agile coach, have you defined the next stage of agile maturity for your organization? In other words, if the organization has already reached from Point A to Point B in their agile transformation journey, what is Point C and what's the roadmap to achieve the same? Agile is a journey not a destination, but there are stages that you go through to continue the journey. Even if you not define a stage per say, there always needs to be some definition of "what's next?".

Every skill set has it's ups and downs. History shows how the demand for technologies has aggressively shifted in last couple of decades. From JAVA, .NET, SOA, Service Bus, Messaging, to MVC, MVVM, Microservices, to Mobile, to crypto-currency mining, to Artificial Intelligence ... and so on. Evolution of profession and skill is the demand of the industry revolution.

In the current VUCA world with complexity in consumer behavior and industry demand, need for agility has never been more. To enable the same, industry experts (call them agile coaches or something else) will always be needed to bring agility at all levels of an organization - delivery teams, org structure, support, strategy, decision making, capability building .. and so on. Patterns will change, demand will vary.

Specifically for agile coaches - is it a black hole moment like a dying star? Or is it a big bang moment of start of new universe driven by past learnings to redefine the scope of coaching with specific skill sets needed to sustain long lasting change? Decide for yourself!

[Disclaimer: All views presented in this article are my own based on my own industry experience. There is no direct or indirect co-relation of these views to my current or previous organizations where I have worked.]