Beyond Scrum and Agile: Blog post #1

This is the first in a series of blog posts examining the worlds of Scrum and Agile from a friendly but critical viewpoint. The authors of The Scrum Guide and the Manifesto for Agile Software Development articulated significant insight into the realities of both product and software development. Unfortunately, many of those insights have become obscured due to arguments over form rather than substance. There is an emphasis on “Scrum practices” and “Agile methodologies” rather than the contexts where those practices and methodologies become effective. At its worst, the emphasis on practices results in teams “going through the motions” without understanding why, and failing to produce results as a consequence. Going through the motions has become so prevalent that the terms “Zombie Scrum” and “Zombie Agile” have recently been coined.

With all due respect to the authors of The Scrum Guide and The Manifesto for Agile Software Development, both document are essentially lists of rules to follow. The message is “Do things this way because it works”.

The purpose of this blog is to explore and explain WHY these practices work. It is to explain the purposes these practices serve and the underlying reasons for their effectiveness. The blog will also try to free people from slavish adherence to the “rules” of Scrum and Agile practice. If practicioners understand the purpose behind a particular rule then they are freed to find alternative ways to serve that same purpose.

I welcome your comments and insights. Please feel free to comment, or to contact me directly with comments and questions.

--Fred Fowler

----------------------------------------------------------------------------------------------

As a top level Agile Coach I tend to get called in when a software organization is in trouble. Most often there is a large scale development effort going on with dozens (or even hundreds) of developers working 10 and 12 hour days writing code. First and second level managers are booked solid with meetings and have no time to spare. There is a frenzy of activity. Millions of dollars (or tens of millions, or even hundreds of millions of dollars) are being spent in order to get the desired results.

Despite this activity and this money, results prove to be elusive. I often get called when there has been a year or more of this going on with little or nothing to show for it.

Oftentimes I find that management has tried to fix this situation by adding more developers to the project. If 50 developers can’t deliver what is desired, let’s make it 100. If 100 developers can’t seem to get the job done, let’s hire 100 more. Let’s make sure they are all busy and working 60, 70 or 80 hour weeks. If we just put enough people to work and work them hard enough we’re sure to finally make the progress we’re looking for.

Fred Brooks, author of The Mythical Man Month coined “Brooks’ Law” by observing "adding human resources to a late software project makes it later.” Very true. The fundamental problem is that the development effort is usually disorganized. Adding more people to the project just makes things even more disorganized. The developers all make sure they are busy all the time, since they are not stupid and do not want to get blamed for the lack of progress. Top management sees all of the activity but cannot figure out why the activity produces no real results.

I once undertook a complete overhaul of the system used by a mid sized distributor of aircraft parts to run its entire business. When I began the engagement their software was unstable and crashed daily. It had been supplied by a vendor that had since gone out of business. There was a need to stabilize the platform and add significant new functionality in order to save the company from failure.

That engagement was one of the most successful ones I’ve ever had. When it was finished the software was bug free and had greatly enhanced capabilities in inventory management, purchasing management, financial analysis, warehouse management and half a dozen other critical areas. More important, the company was profitable and cash-flow positive once again.

I had been retained by the company’s Controller to try to fix the many problems. This all happened in the mid 1990s when the IBM AS400 midrange computer was a popular choice for businesses such as the aircraft parts distributor. Back in those days I was a leading technical expert for that platform and could make it do just about anything I wanted it to do.

On the first morning of the engagement I sat down with the Controller and asked “so what is the biggest problem you need fixed first?”. He had actually prepared a list which he shared with me. The most important problem to solve was a bug that caused a critical batch update process to crash. It was costing the company significant losses.

I set to work and found part of the problem by the end of the day. I coded up a solution to it and then sat down with the Controller just before it was time to go home. I showed him what I had found, what my solution was and the test results showing the solution was correct. He agreed and then together we put the solution into the production environment. With that done, we called it a day.

The next time I came we repeated the process. In the morning he showed me the problem that was the top one on the list. By the end of the day I showed him how I had addressed it. If the solution was ready we put it into production. If not, I showed him the progress I had made and we continued with it the next time.

We followed this pattern throughout the engagement. There was real measurable progress that was made on a daily basis. We tackled the worst issues first and got immediate benefits from doing so. After six months the system was completely stable. After a year it was able to support several new business initiatives. We kept adding value for several years and we finally gave up when there was literally nothing left to change or enhance.

While this engagement began before the legendary OOPSLA '95 conference where Ken Schwaber and Jeff Sutherland first presented the Scrum Framework to the world, the careful reader will note several “scrummy” and “agile” things about the way the work was organized. The Controller was the “Businessman” with which the “Technical team” (me) worked daily. His list of the most critical problems played the role of the “product backlog”. Progress was measured in terms of working software. I had all the skills necessary to work on the IBM AS40 platform with no help from outside, so I was a “cross-functional team”. Our review of the problem list and the selection of the problems to work on each morning constituted a kind of Sprint Planning Meeting. Our review of the day’s results at the end of the day was very much a Sprint Review. The Controller identified the problems but I took responsibility for figuring out how to address them. In effect I was a “self-organizing team”. The Controller gave me the environment and support I needed and trusted me to get the job done.

These days any computer services firm would say it is impossible to completely re-platform an entire enterprise without at least a dozen developers, several architects and a project manager or two. They would also probably say that a team of 50 or more would be needed to get the job done in two years or less. They would be happy to supply all the bodies needed for as much or as little as the client wants to pay.

A good developer can probably produce 300-500 lines of good code per day. Adding people assumes that the bottleneck is the typing speed of the developers. After all, if one person can produce 300-500 lines of code per day, 50 people can produce 15,000 – 25,000 lines per day. Put them to work for a year and they will produce about 1.5 million lines of code (and even more if they work more than 40 hours per week.) More code is better, right?

The Controller and I completely re-platformed the business with no more people than the two of us. Not only that, my schedule had me working for a different client two days per week. I did all my work on the basis of a three day work week.

The key to success for any project, large or small, is to organize the work properly.

Projects in the world of software development are always problem solving exercises. It doesn’t take 150 people to solve one problem. Adding more and more people just makes it more and more difficult for them to figure out what to do. They will end up doing “busy work” because management knows how to measure how busy people are. Management usually does not know how to measure the actual value that is being produced.

The Controller and I were successful with the distribution company because two people is the easiest team to organize. We both accepted our responsibilities and played our roles. The results speak for themselves.

========================================

#scrum #agile #softwaredevelopment #productdevelopment