Why Scrum’s success caused the challenges we now see with it

This is the second in a series of articles:

1. Why Scrum was the original Model-T of Agile
2. Why Scrum’s success caused the challenges we now see with it
3. What’s been learned in the 3 decades since the creation of Scrum
4. Introducing Amplio Scrum
5. Introducing the Amplio Scrum Coach – going beyond the Scrum Master

"Today's problems come from yesterday's 'solutions'" Peter Senge.

“We can't solve problems by using the same kind of thinking we used when we created them."

The Inspiration of Scrum

Scrum was inspired by, and gets its name from, the article The New New Product Development Game by Hirotaka Takeuchi and Ikujiro Nonaka in 1986.

The new way to build products was to recognize we didn’t know everything upfront. That while we could have a goal in front of use, we need to be continuously adjusting our actions to achieve it.

In rugby, a "scrum" is the method of restarting play when a penalty or the ball goes out of bounds or other necessitating event occurs. The term “scrum” is a metaphor for pivoting as we learn new things. The “daily Scrum” reminds us that we are working towards a consistent goal in an ever-changing way.

Scrum has very well-defined requirements, as laid out in the article:

1. You have a cross-functional team focusing on creating a new product.
2. The team regularly adjusts to changing understanding.
3. The team has or can learn, the skills required to get the value created.
4. The team that understands the value to be created.

Scrum Crossed the Chasm, but Successful Scrum Did Not

Scrum’s popularity shifted where it was being applied from where these requirements were met to teams where they weren’t.?

The key requirements for “the new new product development game” to work were met by innovators and early adopters. But not by many who crossed the chasm later.

Why this happened was neither well understood nor thoroughly investigated. Instead, Scrum's proponents explained the failure away by saying “people were failing because they weren’t doing Scrum.” This reflected the belief that “if you did Scrum things would work”.

But this is a tautology – that is, saying the same thing in different ways:

1. If you follow Scrum you’ll get success.
2. If you’re not having success you’re not following Scrum.

This is circular logic – one supports the other – but the rationale of either is not explored.

The difficulty, of course, is that Scrum is based on empiricism without any theory to explain it. It just presumes that if you do it things will work – which they will. But it ignores the issue that you might be unable to follow Scrum effectively.

The creators of Scrum explained its demise when people do “Scrum But.” Ken Schwaber said a "Scrum But" was when "Scrum has exposed a dysfunction that is contributing to the problem but is too hard to fix. A ScrumBut retains the problem while modifying Scrum to make it invisible so that the dysfunction is no longer a thorn in the side of the team"

The onus of Scrum’s failure was thus put on those following it (or, I should say, saying they were following it but actually weren’t).

But whether it was possible or advantageous for teams adopting it to use it effectively was never examined. And there was no understanding of why it worked that would enable it to be changed for the better.

Crossing the chasm (going from early adopters to the early majority) didn’t just represent a change in the thinking of the people now using Scrum. It changed where it was being used as well.

The four cornerstones of Scrum were no longer present much of the time:

1. Cross-functional teams were no longer present because in larger companies there are some teams that support many others. Also, changes to the value creation structure is much more complicated in large organizations.
2. This makes it difficult for regular adjustments since many teams now need to simultaneously adjust.
3. Many late majority teams don’t have a culture of continuous learning. So the skills required to development software this way were resisted and never learned.

The value being created was often not understood because clarity in large organizations is a much greater problem.

In a sense, you could say Scrum died on its own sword.

What was needed was not known

Ken Schwaber described Scrum as a “black box” in his book "Agile Software Development with Scrum." That is, it was based on empiricism and inspecting and adapting. But there was no claim as to what was theoretically happening inside the box. Jeff Sutherland talked about how you can only understand complex systems after the fact.

Adjustments to these requirements and to the roles, events, artifacts, and rules of Scrum could not be made without risk and experiments. And then there was no way to tell where they would work. This added complexity was anathema to Scrum’s foundation and mantras, so it was discouraged by saying Scrum was immutable.

Unfortunately, Scrum described what to do without describing the why of what to do. This further locked people in two ways. First, the thin Scrum Guide was often open to interpretation. Presenting the “why” of Scrum would have created a context for its details – but none were given. See The whys underneath the hows of Scrum for more.

There has long been a misunderstanding that short statements are simple. What we want is conciseness with clarity.

Doing "Scrum By the Book" Is Born

Scrum being immutable is another way of saying you have to do it as it is defined in the Scrum Guide. That is, do it by the book. While Scrum proponents say this is not a good idea, they ignore this is exactly what Scrum being immutable means.

But Scrum “by the book” was created for a particular context. After the chasm was crossed, most teams adopting Scrum were not meeting these requirements.

Saying Scrum worked for complex environments sounded like a qualification of where Scrum works but isn’t. Anywhere you have people you have complexity.

The truth is that Scrum was designed for complex environments. That does not imply that it works in all of them.

Other Side Effects Happened

There was a more insidious problem to Scrum by the book and its lack of theory.

People were told to "follow it to understand."

This, combined with its immutability has led to people following Scrum instead of using it as a framework to think from as it was originally intended.

There again, was no investigation about how Scrum's definition was causing the behavior its creators were ranting against.

A New Path Is Suggested and Rejected

I had been studying Deming and Toyota since the mid 80s. ?When I read the Toyota Production system during this time I realized that the TPS (later generalized and called Lean) could be viewed in a simple way:

1. Focus on creating value for the customer.
2. Take a systems thinking approach as espoused by Deming.
3. Attend to removing delays in your workflow (“just in time”). This requires managing work in process which you could do by using pull methods.
4. Build quality in with automated methods (“use autonomation”)
5. Take a “stop-the-line” mentality (Jidoka and autonomation) to fix errors and their cause as soon as possible.

The key insight was that delays in the workflow and in getting feedback were the primary cause of waste. That Scrum’s “rules” were a reflection of Lean because:

1. Scrum was focused on creating value
2. It had a cross-functional team – while not being systems thinking per se, at least we were looking at the team this way.
3. Scrum’s sprints were formed by pulling from the product backlow and kept WIP within the capacity of the team.
4. Scrum suggested using XP practices.
5. While not “stopping the line”, daily Scrums, demos and retros provided Scrum teams the opportunity to discover problems and solve them.

This is why, in 2008, I suggested that Scrum could be better adopted by considering it a partial implementation of Lean. Unfortunately, Ken and Jeff vociferously rejected this notion, and I was expelled from the Scrum Development group when I brought it up a second time.

This attitude somewhat sealed the fate of those using Scrum outside of where it was originally designed.

Its creators put the blame on those not following it – not looking to see if it could be effectively followed. This had Scrum itself become immutable to its creators.

====?

?

#Scrum #Agile #ScrumFailure #Amplio #AmplioScrum #AmplioScrumCoach