Scrum is Very Confused About Leadership

Scrum is defined by the Scrum Guide, so that is where one should look to learn how Scrum works. Just as the Agile Manifesto defines Agile, the Scrum Guide defines Scrum.

Except that the Agile Manifesto has remained the same since it was written in 2001, but the Scrum Guide keeps changing—a-lot.

Minor tweaks are one thing; large swings in how one of the three major Scrum roles is defined—well that is a problem. It tells me that the people defining Scrum don’t really know what the role should be.

The Scrum Master role was, at one time, defined as the “team leader”. At least, that is my recollection, but I can’t prove it, because the Scrum Guide no longer says that. So my memory could be wrong. But the Scrum Master clearly provides a leadership role for a team. But “leadership” can mean many things.

In 2007 the Scrum Guide said that the Scrum Master “is responsible for the Scrum process, for teaching it to everyone involved in the project, for implementing it so it fits within an organization’s culture and still delivers the expected benefits, and for ensuring that everyone follows its rules and practices.”

In other words, back then, they viewed the Scrum Master as mainly the master of ceremonies—nothing more. The part about “implementing it so it fits within an organization’s culture” is pretty ambiguous, but it sounds to me like it pertains to tailoring Scrum in some way.

But then in March 2007 they added three responsibilities to the role, which I paraphrase here:

1. The ScrumMaster needs to know what tasks have been completed, what tasks have started, any new tasks that have been discovered…
2. The ScrumMaster needs to surface dependencies and blocks which are impediments to the Scrum. They need to be prioritized and tracked…
3. ...the ScrumMaster may notice personal problems or conflicts within the Scrum that need resolution. These need to be clarified by the ScrumMaster and be resolved by dialogue within the team, or the ScrumMaster may need help from management...

Okay, so this adds some pretty specific process responsibilities, akin to task management, which is strange, because Agilists usually prefer stories over tasks. Not a problem really, but seems pretty detailed. Seems like someone noticed some specific issues that needed resolution in the consulting that they were doing that day, and when they got home added it to the Scrum Guide.

Then in 2009 they added,

The ScrumMaster is a facilitative team leader…He must: Ensure that the team is fully functional and productive; Enable close cooperation across all roles and functions; Remove barriers; Shield the team from external interferences; and Ensure that the process is followed...

The “He” is anachronistic—today one would probably say “he or she”, or “they”, but my focus here is the frequency of change.

Then in 2010 they changed the role to be this:

The Scrum Master is responsible for ensuring that the Scrum Team adheres to Scrum values, practices,and rules. The Scrum Master helps the Scrum Team and the organization adopt Scrum. The Scrum Master teaches the Scrum Team by coaching and by leading it to be more productive and produce higher quality products. The Scrum Master helps the Scrum Team understand and use self-organization and cross-functionality. The Scrum Master also helps the Scrum Team do its best in an organizational environment that may not yet be optimized for complex product development. When the Scrum Master helps make these changes, this is called “removing impediments.” The Scrum Master’s role is one of a servant-leader for the Scrum Team.

In other words, they adopted the phrase “servant leader”, but defined it in their own way. It is really important to realize that the way that the above describes servant leadership is quite different from how it is described in books about servant leadership. A servant leader is still a leader, with authority, judgment, and domain knowledge. They are a soft-touch leader, but still a leader.

As of this writing, the Scrum Guide states that the Scrum Master role is responsible for,

• Coaching the Development Team in self-organization and cross-functionality;
• Helping the Development Team to create high-value products;
• Removing impediments to the Development Team’s progress;
• Facilitating Scrum events as requested or needed; and,
• Coaching the Development Team in organizational environments in which Scrum is not yet fully adopted and understood.

The leadership approach of this role is very weak: it is primarily that of a process facilitator. Yet, the most effective leaders I have known were not mere facilitators: they were servant leaders. They kept their eye on things, and they prodded and nudged people, checked in on things continually, and noticed gaps. They had their finger on the pulse of the work, and did not differentiate between technical and non-technical issues. They did not boss people around; but they did not wait for things to happen either: if something needed to be happening and was not, they got people together and said, “What about that? What should we do?” And they did not hesitate to give their own thoughts: they had cultivated the trust of their team so that everyone knew they could speak their mind, and all ideas were discussed on their merits.

Most importantly, while they expect team members to make their own decisions most of the time, a servant leader sometimes makes decisions too, and when they do, they take responsibility for it and don’t blame the team if things don’t work out. They have the team’s back.

To complicate things, the Scrum Alliance defines the Scrum Master differently, and their definition squarely centers around the concept of the Scrum Master being a facilitator of the Scrum process, although it does say that the Scrum Master should be,

Teaching and leading the Development Team to create high-value products...

although that is quite ambiguous—it could mean that the Scrum Master is also a tech lead, and the word “leading” could mean anything from being a dictator to being a servant leader. So I’ll go out on a limb and say that their definition is ambiguous!

The ever-changing and ambiguous Scrum Master role is a major problem. Agile is often blamed today for not working. According to this article in Information Age,

Agile IT in the UK is facing a hidden crisis – 12% of agile projects are failing completely...The truth is that, despite the hype, Agile development doesn’t always work in practice.

Agilists will be quick to point out that “they must not be doing it right”, and that is certainly the case, but if a key aspect such as the role of Scrum Master is ambiguous and always in flux, how can they know what “right” is?

I propose that we throw Scrum out and go back to the Agile Manifesto. That has stood the test of time. Spotify threw Scrum out, and they have their own model of leadership for teams and products. They have multiple leadership roles, and the Scrum Master is not one of them.

It has long been my contention that Scrum does not scale. To make Scrum work in a large program or organization one must add all kinds of additional practices, and usually bend Scrum quite a bit by adding “extended team members” who are technical and business subject matter experts. One usually has to adjust the concept of Product Owner as well, because it is seldom the case that a product has a single stakeholder, so the Product Owner needs to become kind of a spokesperson for them. And most importantly, Scrum does not work for complex products unless one also uses DevOps methods to automate and integrate across teams.

In fact, Scrum is fairly incompatible with the widely used Behavior-Driven Development (BDD) approach, which is much better accommodated by a Kanban process.

But most importantly, the Scrum concept of a team lead, the Scrum Master, is very inadequate for most teams in large programs. There needs to be a tech lead, and there needs to be someone who is making sure the right discussions are happening and that decisions are getting made in real time. That can be the Scrum Master, but if the Scrum Master is not technical, they will not see a large percent of the things that are not getting addressed.

Scrum enthusiasts will say that the team should notice those things, but in my experience, they don’t. They will notice a few things, but they will usually not notice the things that need to happen between teams, and are not happening.

The reason is simple: programmers tend to be introverts. Scrum Masters and Scrum coaches are usually extroverts, and so they often don’t appreciate that difference. Programmers tend to want to keep their head down and do what is in front of them. They would rather do that than run over to another team and ask them about something. In contrast, Scrum Masters often are people who like running over to other teams, so they don’t “get” that programmers are simply not inclined to do that.

And so integration issues between teams tend to not get addressed, unless someone is explicitly assigned the job of watching for those things. In other words, there needs to be a cross-team, product level lead who watches out for integration issues. Scrum says nothing about that. A “Scrum of Scrums” can help there, but it is usually insufficient in my experience.

The inescapable conclusion is that Scrum is a very poor model for how teams should work—and work together—on large Agile programs building complex things. It changes and its leadership model is deficient.

And we don’t need Scrum. What we need is the right model for leadership, which I contend is servant leadership—real servant leadership, which is not what the Scrum Guide describes. Real leaders need the freedom to define their own program structures and cadence based activities that will ensure that the Product Owner is able to define features and get them built and deployed in a timely manner and the features work, and the entire process is informed by Agile principles, DevOps ideas, Lean ideas, 12-factor principles, and many, many other ideas. They just need the freedom to think for themselves, instead of yielding their brains to the Scrum Guide.