The Heart of Agile is Missing

There is something critical that is missing from your Agile transformation. You can't buy it, you can't hire a consultant to implement it, you can't install it in your software delivery pipeline. And if you continue with your Agile transformation without addressing it, you are virtually guaranteed to not only to fail, but probably to be further behind than you were when you started on your Agile journey.

Let me explain.

Heartbeat

I don't remember exactly when I first tried pair programming. But I do remember one of the first times. It changed my experience of software development forever. The power of that experience illustrates the main reason why Agile works so well when it does. But that reason, I suspect, is not what you think.

We were at a cafe in the Mission in San Francisco, and it must have been around 2005. I ran a software dev shop back then, and I'd secured a new contract for software project. It was too big a project to do alone, so I had hired a friend, let's call him Jim, to help me work on it.

When I started coding in early 2000, another friend had given me a copy of Kent Beck's eXtreme Programming. I was deeply influenced by it. Jim was a fan too, and he and I were now very eager to put as many principles from that book into place as we could for this project.

We met regularly with the client, sharing with them our feature ideas and early rough examples of working code for feedback (iterations). We'd set up a unit testing framework that ran automatically whenever we made a change to the code (CI/CD). And we both committed at the start of the project to follow test-first development principles as much as possible (TDD). There was a little cheating here and there on the TDD, but for the most part we stayed true to that method.

The one principle that eluded us the longest, simply because we did most of our programming work from home, was pair programming. It wasn't practical for us to pair program for the entire project, but we committed to getting together a couple times a week in person to work on a few features. And those afternoons were an absolutely joyful experience.

The code we produced for that project was the best I'd ever contributed to up to that point. But that wasn't the reason for my joy. Yes, I was proud of the craftsmanship in that software system, the ease at which we could make any change whatsoever and get instantly notified by a set of passing tests that we'd not broken anything. That the code was clean and modular. All of that was great. But it was not the source of our joy. That source was something else entirely.

Heartless

I've seen a lot of companies engaged in activities that they claim to be some sort of Agile transformation. Very few of them are experiencing the benefits they'd hoped for when they began.

It's not a lack of discipline that eludes them. Many are quite disciplined in their approach to change management. They send droves of employees to Agile trainings, and deploy armies of consultants and coaches to mediate the process of change. They install online training materials. They make pronouncements at company all-hands about the glorious new world that is just around the corner.

And yet, day after day, the grinding reality of the corporate status quo seems more entrenched than ever. "Where is the glorious future you promised," employees begin to ask?

Over time, arguments break out about whether we're "doing Agile right." Some push for strict adherence to the original documentation. Others give up entirely, writing off this whole Agile thing as yet another change management fad, a version of innovation theatre that allows senior executives to look like they're actually doing something new for the Wall St analysts' pleasure. These disillusioned folk embark on their own subversive version of change, often incurring the wrath of the senior leadership when their rebellion is inevitably discovered.

Both the rigid adherents and disgruntled rebels are right in their own way.

We need not quote scripture at each other from the Kent Beck's book or the Scrum Guide in order to discover the heart of the Agile. The words, diagrams, definitions, and documents that have been produced by the various Agile sects over the years are not some sort of magical incantation where if we only faithfully adhere to their letters precisely, our wishes with be granted by the gods of continuous improvement.

Nor is it prudent to ignore and abandon the teachings of those who've struggled before you. Reinventing the wheel all the time, because you're too proud to learn from others, is a great way to fall irrevocably far behind your competitors.

Clearly Agile dogma serves no-one. We believe in learning from mistakes, and evolving methods to handle new realities in the world. But there is something foundational of value in Agile methods that must be experienced first hand in order to be successfully applied to your organization's transformational mission.

The Human Touch

Taiichi Ohno, the father of the Toyota Production System, in his memoir of the same title, used a curious phrase to describe one of the original two pillars of the system: autonomation, or "automation with a human touch." The idea is that machines do not simply run continuously without monitoring by a person. Rather the machine is imbued with fault tolerant mechanisms that alert a human to the problem as soon as it occurs. It is the human that solves the problem, usually then updating the process to avoid it occurring again.

This idea dates all the way back to the original automatic loom designed by Toyoda Sakichi, founder of the Toyota Company, whereby the loom would stop running as soon as the thread was broken, so that the weaver could intervene and fix it. This same concept is visible in any modern CI/CD pipeline or automated testing framework.

Over time, the Toyota Way was updated to redefine its two pillars as "Respect People" and "Continuous Improvement". Respecting people is a deeper concept than autonomation, and one that was forged in the real-world experience of the continuous improvement movement.

A contemporary of Ohno's, W. Edwards Deming, whose work is also credited with kicking of the continuous improvement methods was a fierce defender of the empowerment of the people doing the work to find new ways of working that not only raised the quality of output, but also made their work more enjoyable. After all, happy workers produce better outcomes.

Empowered Teams

And of course these principles are readily visible in Agile. In fact, each of the major flavors of Agile (XP, Scrum, Kanban) were heavily influenced by the continuous improvement movement, and specifically Toyota. Kent Beck, Jeff Sutherland, and David Anderson each have repeatedly called out Toyota's work, and Lean in general, as the inspiration for their unique contributions to the Agile movement.

The foundational principles of Agile that make it effective are actually all about human interaction and basic psychology, and not really about technology at all. Things that look like engineering principles are actually about the human brain and our creativity and focus.

For example, here are two great snippets from the original eXtreme Programming website that make the point brilliantly:

“The most important thing on a project is good leadership; the least important thing is who leads.”

To be effective, leaders need to put focus on those being led, not the leader themselves. Good leaders know this. The selflessness of leadership is a manifestation of respecting people.

And then there is this gem:

“Hermits don’t share, communities are based on sharing.”

Agile is about teams. And team members share with one another. Are there people in your organization who seem to be keeping information to themselves? That would be an example of not respecting people.

The Scrum Guide, too, provides great guidance here. Recall the pillars of Scrum: Transparency, Inspection, and Adaptation.

The guide says that “adaptation becomes more difficult when the people involved are not empowered or self-managing.“

Despite its subtle and uncontroversial language, this passage basically screams out that if you dominate and control your teams they will not be able to adapt. Are your teams empowered and self-managing? Before you answer, why don't you go ask them.

And my favorite has to be from the Kanban method. One of the key principles of Kanban is:

“Manage the work and let people self-organize around it.“

So, having a Kanban board isn't enough? Measuring cycle time and enforcing work-in-process limits not sufficient? Afraid not, friends! What really matters is team self-management. Skip that step and you'll never make it to the glorious future. And the only way a manager can allow a team to self-manage is by trusting and respecting them.

Flow and Joy

As Jim and I first pair-programmed together on that codebase way back in 2005, we both felt the joy of working together as a team, the joy of collaboration. We both felt a sense of respect and admiration for each other, and a sense of companionship that comes only from working with other people.

We both knew, without ever having to express it, that neither of us held all of the answers for the perfect solution in our heads. It was only through our openness and collaboration, undertaken willingly and without fear of judgement or embarrassment, that we were able to produce an outcome that was far superior to what either of us could have produced alone.

Since those days, I have worked on countless technology projects. The successful projects stand out in my mind because those were the projects where we felt the most joy as a team. We respected each other, we shared knowledge, we admitted our mistakes freely and celebrated our wins together.

What makes this type of collaboration possible? Can it be recreated?

Modern research into human behavior illuminates that most of what is good about Agile actually emerges from good goal setting, fast feedback loops, and most importantly, developing empathy with others, be they your team, your customers, your stakeholders.

Empathy is at the heart of Agile. Without it, we are only using automation without a human touch. The outcomes then are cold, dreary, and unsatisfying.

Listen To The Heart

You are embarking on an Agile journey because you want to make a difference in the world. And now the pandemic has caused you to rethink the relative importance of everything on your to-do list. Suddenly things that seemed so important a year ago no longer do, and there is a deeper motivation you were only scarcely aware of before. You're not satisfied with mediocre and humdrum. You're ready to do something different.

Apply that newly critical eye to your Agile work. Here's how you can approach it afresh:

1. Check your own biases and baggage. Everyone--including you--is operating from a place of bias. We all have our preconceived notions of how we perceive the world and make sense of it. Before you set out to change others, look first deeply within yourself and make sure you're also open to embracing change.
2. Understand your people. Most of the time we don't pick our colleagues. We work with a group of people who have different strengths and weaknesses, values and goals. Make an effort to get to know everyone on your team in a more personal way. Find out what their values are and have explicit conversations about which values we share as a team.
3. Look at the way you're working now. Where are the bottlenecks and breakdowns? Wherever you find areas of conflict, focus your attention there first, before trying to change anything else.
4. Read the principles from each of the Agile methods, identify those that seem to address your particular problems. You don't need to follow any particular method by the book. It's a lot better to choose methods that align with the team's pre-existing values. Changing process is a lot easier than changing people.
5. Work together in a group to decide on small things to change. Most teams stumble in the beginning because they try to change too many things at once. Limit your changes to one key factor each iteration.
6. Work in teams to experiment with working in a new way. Treat your process as a laboratory experiment. Each iteration be clear about what you plan to improve and what measurements you will use to decide whether or not it worked.
7. Finally, look at what worked and what didn't in your experiment. Share your findings openly. Share the results of your experiments with other teams in your organization. Publish blog posts about it. Solicit feedback from everyone you can. Have courage to learn and grow together as a team.
8. Repeat the process at step 5 (and occasionally revisit steps 1 - 4).

Working with people together can be incredibly joyful. We just have to be open and trusting enough to do it for real.

My passion is to make work better for all people. When organizations are dysfunctional, the negative effects on society are massive. People in the organization are miserable, customer needs are ignored, and we all have to deal with sub par products and services. Employees go home with fear and frustration in their hearts, and make life miserable for their families and communities.

We can change it. If you really approach Agile from a people-centered view and not a process or tools view, you will find exponential improvement rather than just linear or even negative progress.

--

Ready to improve technology leadership in your organization? Try our free diagnostic tool .