Agile: Avoid the Hype, Use Only What Works

A colleague of mine recently posted a vlog describing how many companies’ top-down efforts applying agile software project management, under the guise of improving processes, too often end up with the opposite result: late projects, defective software, and burned-out developers. Having practiced elements of XP, Scrum, and Kanban at different companies over the years, with varying degrees of success and failure, I thought I’d put down my thoughts, experiences, and ideas here.

Recently I’ve noticed a sharp increase in pushback against agile/scrum rituals and ceremonies, some going so far as to declare them invalid and completely useless. I’ve seen my share of agile efforts gone wrong at a few? companies throughout my career. In the places where it went poorly, it was treated more like a religion than a set of guidelines and best-practices. Usually, some authority would dictate the tools and procedures that we all had to follow. However, schedules often slipped, standups were a waste of time, and the teams were reluctant to truly buy into the methodology; just going along with it to avoid conflict.

But I’ve also had experiences with agile that went well, and actually improved teamwork, productivity, and estimation/scheduling. In those cases, company leaders gave us the freedom and responsibility to determine how we could improve process and communication ourselves, rather than assigning a certified scrum master (CSM) to “enlighten” the team with agile dogma.? We chose our own set of techniques based on trying and keeping that which worked best. I don’t believe agile principles and techniques are necessarily bad, but I'm pretty sure that forcing dev teams to participate in rituals that don’t really help them doesn’t work.

What is Agile Anyway?

It is important to distinguish between agile principles and agile frameworks. The Agile Manifesto, lists 4 basic values and 12 principles drawn from the founders’ experiences, that they believe can help lead to better software development. Whereas, the frameworks, like XP, Scrum, Kanban, etc., are sets of procedural techniques that have been promoted as specific ways of applying the values and principles, and which practitioners assert will give good results when adopted by software development teams. Big problems can occur when teams focus on the techniques without thinking much about the principles, and how they may or may not apply to their particular software development situation.

TLDR, if you’re concentrating on rituals and ceremonies, without considering the values and principles of agile, you’re doing it wrong.

Will Agile Work For You?

Before considering any agile-based approach for your team, it is critical to have a clear understanding of the environment, business goals, and basic organization of engineering, management, and other stakeholders. Only with this insight can you effectively apply agile principles, values, and even techniques to your team’s situation.

As an example, carefully look over? the following 4 values taken directly from the Agile Manifesto:

1. Individuals and interactions over processes and tools
2. Working software over comprehensive documentation
3. Customer collaboration over contract negotiation
4. Responding to change over following a plan

Now, with only those 4 values and a good understanding of your team’s environment, you can determine how much agile may help (or hurt) your efforts to project manage and deliver software. If for instance, management prioritizes plans and/or contracts, and your team is focused on deadlines, tools, processes, and documentation (over say, automated testing & continuous deployment), agile alone may not work well. No amount of user stories, sprint planning, burn-down charts, velocity measurement, or other agile techniques will make up for a lack of customer/stakeholder involvement, commitment to correct software, good communication among engineers, or plan flexibility. If you don’t have these things, your team is simply not agile with respect to the manifesto..

That’s not necessarily as bad as it sounds–the reality is that businesses often do need to set aggressive deadlines, comply with contractual obligations, documentation requirements, industry standards, and follow some form of long-term planning. Things that companies actually get paid for may have higher value than, and can even conflict with the agile manifesto values and principles. Some agile procedures, even partially adopted, can help. But others may just get in the way. I believe it is a mistake to assume that by just adopting agile procedures, you will produce all the claimed benefits of agile.

Less Is More, Start Out Small

I’ve found that software teams work best with a minimal amount of administrative overhead tasks; software developers are most productive when working on software, not when estimating stories, sprint retrospectives, generating reports, or spending time on many of the other agile ceremonies. Too often, CSM types will spend an enormous effort architecting workflows in Jira to “scrumify” a loosely organized engineering team–only to end up slowing the team down and wrecking morale. Agile ceremonies like sprint planning, daily standups, and sprint delivery can be useful. But those and others, like story estimation, backlog grooming, sprint review, velocity tracking, etc. may take time to get the team up to speed, and some may not even end up being needed at all.

How Much “agile” Do You Really Need?

If your standups drag on with multiple deep dives and turn into 1-2-hour time-wasting meetings from hell, that’s an ideal place to start optimizing. Try enforcing true 15-minute standups, quickly reviewing the day’s tasks rather than giving the floor to each person in succession. If team members want to deep dive (they always do, and that’s a good thing), try to get them to save it until the main task review is done, or better yet, have them do separate break-out meetings.

Maybe you don’t even need daily standups–if you have active chat channels or mostly ad-hoc in-person gatherings when someone is blocked, that may work better than subjecting the whole team to a daily workflow interruption.? Perhaps start by getting the team used to recording tasks, stories, bugs with the right information first. See how that goes before adding sprints w/planning and retrospectives. If you manage to implement reliable CI/CD, you might not even need sprints; continuous Kanban may be more suited for your team. Try to assess task estimation and velocity and determine if it is accurate before committing to aggressive delivery schedules and depending on “agile” to work it out.

Conclusion

Most scrum documentation and training I’ve seen concentrates on the many (too many!) techniques and procedures with very little coverage of whether they apply, or how they can go wrong. I believe this leads too many project managers and CSMs to act like evangelists rather than engineers; promoting rituals and ceremonies with little regard for whether or not they’re actually helping. I’ve actually seen well-intentioned leaders, managers, and CSMs fall into this pattern at some companies. It is totally avoidable if you can stop thinking of agile as a “must have” and treat it as “something that may help”, and trust yourself and your team to figure out what works best for them. Once of the basic agile principles is about delivering incremental value in small successive iterations. That should be applied to the adoption of agile itself; it's more likely to succeed than any top-down approach.