Scaling Agile: Scrum and beyond

All water fall shops are alike; each agile shop is agile in its own way.. If Tolstoy was a modern Agile coach, he would be the first person to declare that.

A probable reason for the difference is that each organization is trying to solve for Scaling Agile while subconsciously staying true to Conway's law, 'Any piece of software reflects the organizational structure that produced it'. After talking to several agile practitioners from various companies, one thing is for sure, there are as many versions of agile as there are companies.

To understand this we could perhaps start from the beginning; back to the founding principles of Agile. The Agile manifesto was created by developers who practiced light weight development methods from the 90's like Xtreme programming, Scrum, Unified Process and Dynamics systems development method.

The manifesto created in 2001 states the following:

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

•  Individuals and Interactions over processes and tools
• Working software over comprehensive documentation 
• Customer collaboration over contract negotiation
• Responding to change over following a plan

That is, while there is value in the items on the right we value the items on the left more 

 The organizational biases towards the left (Individuals and Interactions, Working Software, Customer collaboration and Responding to change) and the right items (Processes and tools, Comprehensive documentation, Contract negotiation and Following a plan) could explain the differences between Agile implementations.  For simplicity sake let's call them Left Bias Agile and Right Bias Agile organizations.

The basic agile concept is an easy sell for developers. Work on your craft, talk directly with the users of your software, don't obsess about documentation and don't even worry about getting it right the first time. That's pretty likable. For most developers this is the way software development used to be and ought to be. For companies created in the agile age and developed on agile principles this has to be close to the nirvana of software development. 

The same might not be the case for larger organizations with slightly more ancient technology, say older than the 2010's.

Their challenges could include

• Bridging of legacy technology with the dream of the seamless digital customer experience.
• Differing agile adoption change curves between IT and business.
• Not to forget, execution of large initiatives with aggressive dates, multiple dependent constituents, a mix of legacy systems and latest technologies, external parties, geographically disbursed teams, architectural road map journey; topped up with a highly regulatory environment.

Now.. that's when the theory starts breaking down and cracks appear in the initial purist agile zeal of the organization. 

Based on conversations with various folks from large enterprises, there are two main approaches that seem to be favored for waterfall to Agile transformation. Let's give it simplistic jumping-into-water analogies to go with the water theme.

The life jacket approach (Right Bias) where you develop all requirements break them into features, then using scrum to develop the features in two plus weeks sprints. When the work is done, have a testing organization test everything to perfection before going live in production. 

The leap of faith approach (Left Bias) which could use frameworks like SAFe (Scaled Agile Framework) or Scrum of Scrums to convert the organization into agile very fast. Product managers/ owners are responsible to create epics and features. Testing is done using a combination of mostly manual and some automated processes. The rest is up to empowered teams and individuals who might go through a near death experience or two before starting to develop techniques to handle large initiatives. 

Both of these have their advantages and disadvantages. Most organizations are probably somewhere between these two approaches. 

The Life jacket approach gives the organization a better sense of control over the situation and gives better visibility into the initiatives in the portfolio. It also probably gives better ability for leadership to help if the teams need it.  This fits well with the Right Bias organization that values the discipline of processes and overall visibility. But from an agile journey perspective the organization will probably be slow on the take towards business agility and rapid innovation, the ultimate goal of agile. 

The Leap of faith approach is probably be the fastest way to move an existing organization into agile; it's also a good way to figure out agile adopters, evangelists and leaders in the organization. It takes a little time before teams genuinely understand the meaning of empowerment, but once they get it, they can fly with it. When you are looking at the portfolio level it is a little chaotic because everyone's running on a different rhythm and it is tough to get a good handle of what exactly is happening at any point of time. The middle level leaders will most likely feel lost as they don't know what their roles are any more and the product organization have a lot more project management and testing on their plate than they thought they had signed up for. This fits with a Left Biased organization, producing innovation, but could result in quality issues and slight dizziness in the shop.

Being someone who learnt swimming as an adult(and not very proficient) and still wary of the sea after watching JAWS as a 10 year old, I personally favor the Support Boat approach. I came across this while reading about a totally different subject; swimmers preparation while crossing the English channel. This is a situation where you have highly motivated and trained swimmers supported by a boat and personnel that accompany them through the entire journey.

To add more clarity to the concept, here is an excerpt from the English Channel nycswim.org FAQ website for channel swimmers. 

Will you have an escort boat?

There will be an escort boat right next to you from which your crew will give you hot liquids and encouragement. The boat pilot will plot the course during the swim.

Usually you will swim no more than 10 to 20 yards away from the boat.

Who will need to be on your escort boat?

A coach, training partner, spouse/parent, an official observer of the Channel Swimming Association and the boat pilot.

Do you need a coach?

Ideally, yes but many swimmers have been successful implementing their own well-thought out plans.

What does your crew/observer do?

The crew will monitor your condition for fatigue and hypothermia, feed you periodically, watch for boats, alert you of debris, and keep you as informed and entertained as you need to be. They will also keep a log of the swim and promise to take lots of pictures. You should have worked together for a few years so they know what your normal condition should be at various points of a long swim.

Can you stop at all?

Yes. You can stop whenever you want. You will probably be stopped every 30 minutes for about 30 seconds to get a "feeding" of hot liquids administered from the boat by a "feeding pole" (supplied by you; usually a long pole with a cup holder at the end). You can't touch anything or anybody except this pole. The more you stop, the slower your overall time will be, but worse, you could miss catching a tide that might push you towards France.

Are there any sharks in the English Channel?

There are no self-respecting sharks in the English Channel. It's too far north and too cold.

The support boat and crew concept works very well for the swimmer, as the swim could take anywhere from 8 hours to 26 hours dealing with cold salt water, darkness, jellyfish, sudden shifts in water, wind, waves, fog or a cold-loving, freak-of-nature shark.

This concept was in my mind while researching some new Agile frameworks that incorporate more traditional IT leadership principles, when I came across the Disciplined Agile Delivery (DAD or DA) framework for the next level of Agile scaling.

The framework extends scrum concept with strategies from Agile Modelling, Xtreme programing, Kanban, Lean Software development and other techniques.  Wikipedia calls it " a second-generation framework that strives to provide a coherent, end-to-end strategy for how agile solution delivery works in practice. DAD is a people-first, learning-oriented hybrid agile approach to IT solution delivery. It has a risk-value lifecycle, is goal-driven, is scalable, and is enterprise aware". 

At least on paper, this seems like a great option to scale Agile to run the whole of IT, not just the IT initiatives. A lot of the issues that scrum might not address directly, like deadlines, dates, macro level visibility to initiatives, mega scale initiatives and so on might have a solution by combining agile principles with back to basic IT management principles.

This could be where we organizationally understand the problem that needs to be solved first and then form teams and techniques around it as opposed to trying to fit all problems within the same size framework. 

This might also help us from reinventing the wheel in every agile shop and instead, spend our time on producing awesome software.

 I am curious to hear from other Agile practitioners on your experiences with large scale initiatives using Agile and what techniques and frameworks you have found useful. Is there some framework out there that works really well?