When Scrum is not Scrum

My book, Unlocking Agile’s Missed Potential and my LinkedIn articles have a common theme – Agile has not met expectations because of the constraints of waterfall planning. I’m going to address Scrum specifically in this article. I’ll show you that Scrum is not possible when teams are held to feature schedules derived from waterfall planning. And I assert that “real” Scrum is only being done in a fraction of the software industry, mostly in smaller companies. My conclusion is that much of the criticism of Scrum you see isn’t valid because almost nobody is really doing Scrum. Don’t throw the baby out with the bath water!

I assert that there are a few key practices in Scrum that must be there to say you are “doing Scrum:”

1.????Teams pull work from backlogs at their own pace

2.????Feedback is implemented at a higher priority than new development

3.????Defects are corrected at a higher priority than new development so that releases remain “potentially shippable”

4.????Sprints produce working functionality, not chunks of software

5.????An engaged product owner can convey the problems to solve for the team

Arguably there are more criteria, like acceptance test plans, and sprint lengths must be 2 – 4 weeks. But the first three criteria are sufficient to demonstrate that Scrum has not been done in most organizations.

My logic goes like this. We know that we could never estimate software effort and schedule in the waterfall days to reliably meet feature-level commitments, even with good requirements. We ended up throwing out partially developed features near the end of the release or extending the schedule. Many Scrum teams must commit to features in a release before requirements exist. Add the additional uncertainty of how much feedback will need to be implemented during the sprints, or how many defects need to be corrected. How could you possibly create an accurate estimate for a release containing hundreds of features?

In 10 years of consulting, I heard many organizations say they were doing “hybrid Agile” with fixed release schedules and features. This violates my first three Scrum criteria. In these organizations, team sprint backlogs must keep pace with feature schedules. Teams struggle to complete “sprint commitments” by doing the minimum to get a “done” checkmark at the next sprint review. Feedback? Implementing feedback takes away time to meet schedule commitments for the next sprint. Defects? Pile them up and hope there is time at the end to fix the visible defects in a “hardening sprint.” Teams feel like losers because they’re always chasing goals they can’t meet.

Now you’re probably thinking that the logical solution is to include development effort contingency to account for software estimation variance, feedback, and defects. But it doesn’t happen. This relates to the “dirty little secret” revealed in my recent article, Why the Business Needs Predictability from Agile. Most businesses use pressure to “motivate” their software development organization. “If we let them have contingency, they won’t work as hard.”

So, the conclusion is that Scrum has not really been done in most organizations because the first three criteria are violated. The first step in achieving the Scrum vision is to recognize this, instead of trying to force fit Scrum into fixed schedule and content planning. I’ve usually seen Scrum taught as if the first three criteria can be assumed in your organization. When concerns are raised, the instructor defaults to, “Your business is just going to have to become more agile.”

The teams go away and pretend to do Scrum, hoping one day their stupid management will finally grasp the benefits of business agility. Of course, your management understands the benefits of agility, but they stick with waterfall planning in a vain attempt to get the predictability they need. Scrum has not provided an alternative.

The fourth criterion is that sprints produce working functionality instead of chunks of software. This is also violated in most organizations. I see requests for functionality disguised as user stories. Sprints produce small increments of software functionality instead of demonstrable user functionality. “As an accountant, I want a drop down to select the region.” ?

Many believed they could just start “doing Scrum” because the basic principles are simple. True story. I was doing an assessment for a large well-known organization. The engineering department had “gone Agile.” I asked one of the engineers what they had changed to declare they were now Agile. The answer, “We’re doing standups and writing less documentation now.” Another classic comment from a development director at a different organization. “We’ve pretty much got Scrum down now. We just need to figure out the product owner role.”

The first point of this article is that Scrum can never be real Scrum unless waterfall planning is replaced by agile planning, like what is proposed in my book. Even then, organizations need to determine what they believe are the fundamental practices and ensure they are applied consistently. Of course, the Agile framework itself and the practices must be based on project types in which the criteria can be met. For example, you may find that Kanban is a better alternative to Scrum for some projects.

The second point is that Scrum has missed expectations in most organizations because it’s not really being done. It’s “hybrid Agile” with many of the practices cherry-picked by individual teams. Real Scrum can meet the expectations of the business and developers.

Criterion five about having an effective product owner? That’s the topic of my next article, “The Product Owner is a Unicorn,” another problem that needs to be solved for Scrum to achieve its potential.

Would love to hear comments on the extent to which this article resonates with what’s going on in your organization. And feel free to forward to others where Agile is constrained by waterfall planning.