Agile Transformation: What “Culture Change” Means

Some time during the early part of the last decade - the 2010s, which we have just left behind - the Agile community, struggling against accusations that Agile does not scale, had the collective epiphany that an organization’s culture is a governing element in the organization’s ability to utilize Agile methods.

It did not matter that the Agile community, by and large, did not have a clue what “culture” was. Culture was the perfect scapegoat: Agile wasn’t working in big established organizations - and the culture was to blame. Blame shifted - problem solved: Agile was not the problem - the organization was the problem: the organization was in its own way.

Culture was an invisible enemy, a boogeyman, and it enabled Agile coaches to shift the blame to their clients! - like a chef saying, “You don’t like my cooking! - then you have bad taste!”

Culture was a great scapegoat because no one could deny that organizations have culture and that culture is powerful, but what part of culture was to blame for Agile’s difficulties? The Agile community did not know: their answer was the almost comically ambiguous prescription and catch phrase, “Don’t do Agile - be Agile”.

Seldom have I heard a less actionable piece of advice. I found the situation so frustrating that I created the online magazine Transition2Agile.com. For that effort, I brought together thought leaders in organization culture, as well as world class Agilist practitioners. One of our articles hit the culture question head on.

You see, as an early adopter of Agile (in 2000) I am part of the Agile community, but I have long been troubled by the dogma and impractical advice that comes from so-called thought leaders in the community. Most Agile advice does not work at scale and is too extreme or simple-minded for what large organizations have to consider, even though the core principles are extremely powerful.

The problem with blaming culture is that culture is the very hardest thing to change. It is like saying, “We want to read books, so first we need to create a library”. Because culture change is so hard, organization culture expert Eric Eyl says, “Stop trying to transform culture - build on existing strengths”.

Proponents of focusing on culture often quote Peter Drucker who famously said, “Culture eats strategy for breakfast”, but while that is true - culture is indeed the determinant of an organization’s natural behavior - it is also true that culture follows behavior: if you change key behaviors, and the behaviors are successful, the new behaviors will become part of the culture, and they will help to shift the culture overall.

In other words, behavioral change is a driver of cultural change. That is the entire basis for cognitive behavioral therapy: to change key behaviors so that, over time, those behaviors become ingrained and become the new normal. As a result, attitudes, perceptions, and self-identity change.

Yes, we need to change the culture, but we cannot start from there: that is too large an ocean to boil. We need to be smart, and focus our efforts on things that will eventually catalyze cultural change.

In thinking about how an organization changes during any kind of transformation, it is useful to borrow a metaphor from physics. Hold on: we are not going to get too technical. In physics, a “state change” is when something - say water - changes from one state, say liquid, to another, say ice. The change does not happen all at once: what happens is that small pockets of ice form, and begin to grow. As the pockets of ice increase in size and number, their influence increases, and the rate of change increases.

This metaphor is not mine: it was described by biotechnologist and physicist Safi Bahcall, who was a member of President Obama's Council of Advisers on Science Technology. Bahcall was talking about how innovation occurs within society.

In a similar way, organization change needs clusters of change that spread. Change does not happen all at once, in lock step. It needs to be seeded, and be successful, which encourages other clusters to form, and soon the clusters are so numerous that they are part of the new normal, and so adopting the new behavior is no longer risky, because “others are doing it”.

Perceived risk is the main reason why people resist change. Most people in an organization prefer the status quo, because they feel safe that way: their job is safe, their status within the organization is safe, and it is easy: they do not have to think or learn anything new to keep doing the same thing. And most of all, they do not want to be the one to “go out on a limb”, risking failure and consequent loss of status.

Yet risk is an unavoidable element of change, particularly change that requires a paradigm shift. That kind of change entails creative thinking. Creativity is about thinking of new approaches - new paradigms - but creativity is risky. The musician Sting has said,

“How would I define creativity? For me, it’s the ability to take a risk - to actually put yourself on the line, and risk ridicule, being pilloried, criticized or whatever, but you have an idea that you think you want to put out there, and you must take that risk.”

Thus, for transformational change, your creative thinkers and your risk takers are the early adopters who form the first clusters of change. You need them: change will not happen without them. Most people will not take the risk - only the creative thinkers will: those who are not content to keep doing the same thing.

A warning: when change is underway, it seems like confusion: during change, there will be different views expressed about how things should be done. That is the process of a new consensus emerging. For a complex change such as Agile and DevOps transformation, it is not possible to plan it all and then “roll it out”. A successful Agile and DevOps transformation is messy.

The reason for this is that an Agile and DevOps transformation is primarily about everyone - at all levels of the organization - learning new methods and new ways of working. Many of the methods are complex and have tradeoffs - so there is no “best way”. As a result, a-lot of discussion is needed, and disagreement is a common element of discussion.

To those who prefer status quo and stability, the multitude of viewpoints are disturbing and will seem like confusion. You might even hear someone say, “They confused my staff by telling them something different from what others told my staff”. But that discord is normal during transformation. It is not bad - it is good. The staff will have to think for themselves, instead of blindly following what someone told them. They will have to try to understand at a deeper level, and ask questions - become part of the discussion. Eventually a new normal will emerge, and harmony will return somewhat, although not entirely: today, change is becoming the new normal.

Transformation can stall, or even fail, if the risk-averse people manage to keep things the same long enough that the transformation starts to look like it is more trouble than it is worth. Most of the organization will try to keep things the same - to change in a superficial way but still do things the same underneath. They will try to “map” the “new way” to the “current way” - but for an Agile or DevOps transformation the changes are so fundamental that such a mapping is not possible.

The risk-averse people will shut down healthy debate as “confusing” and insist that there be a single plan for change, with all of the details spelled out up front: without that, they claim, they cannot tell their staff what to do.

Yet in an Agile and DevOps transformation, you cannot do it that way. There can be no detailed up front plan. It is possible, and very valuable, to create an initial set of strategies, but one cannot go much deeper than that at the beginning: the details need to emerge over time, through people trying and sharing what they do - learning in the process. Over time, the organization accumulates an ever-evolving set of common “practices” and “patterns” for “how to do Agile and DevOps here”.

This is why executive leadership is needed: there can be no up front detailed plan to “roll out” Agile or DevOps, and so someone needs to drive the transformation process: someone with enough authority to keep it a priority long enough for it to take root. The dilemma is that in order to drive the change, one must understand Agile and DevOps. Otherwise, there is a large risk of driving the change in the wrong direction and making colossal mistakes.

Ultimately one wants to change the culture: change is self-sustaining when the culture has changed - by definition. My main point here is that to change culture, you must change some behaviors first. Some executive-driven mandates are often necessary - hopefully mandates that are not colossal mistakes - so that people give change a high priority; otherwise, change will be put aside as soon as the next operational crisis appears. And executives must get into the discussions - not leave them to staff.

Executives must learn enough about Agile and DevOps so that they can drive the changes intelligently. Learning just a little bit is dangerous. Knowing Agile and DevOps at the catch-phrase level is not sufficient, because catch-phrase Agile is almost always wrong: the catch-phrases promote extremes that do not work.

My favorite popular catch-phrase today is “autonomous self-organizing teams”. That does not work: if you are going to go that route, what works is mostly autonomous, mostly self-organizing teams, that are supported by a well funded self-service tools group - something that usually takes Netflix’s level of IT resources to do right. Thus, you will need to apply judgment to know what “mostly” should translate into, and also judge whether your level of funding can create an effective self-service tools group.

My favorite popular catch-phrase today is “autonomous self-organizing teams”. That does not work

The biggest mistake that Agile transformation executives make is when they assume that Agile or DevOps transformation is like other transformations they have overseen before - it is not. It is deeper, I guarantee it. Another mistake is assuming that Agile and DevOps are merely another variant on old IT methods. They are not: they are deeply different. Some individual practices have history, but the way it all works together is new - and I am an old-timer saying that.

To learn enough to drive an Agile and DevOps transformation, leaders need to read the best published material. Beware articles written by people who are pushing catch-phrases. The approaches promoted in a large percent of the articles about Agile and DevOps only work at small scale, such as a single pipeline for a one-product organization.

Leadership need to have deep conversations with staff to think things through. My strongest advice to IT executives in an Agile or DevOps transformation is to clear some of their calendar and make time for conversation. Meet with key SMEs, without a meeting agenda, to talk through concerns, for a minimum of an hour at a time. Half hour discussions cannot go deep, and deep is the goal.

Meet with external Agile and DevOps experts in the same way - preferably independent ones who are not trying to place staff. And remember that Agile and DevOps look different when you do them at scale with multiple integrated products versus in a single-product startup, and also remember that what works for revenue-rich monopolies such as Google does not necessarily work for other companies, so make sure that some of the experts you talk to have Agile and DevOps experience in a range of situations.

When you arrange conversations with people, tell them not to prepare a slide deck - slide decks are toxic for deep thought. Also tell them to not prepare an “ask” - that you merely want to talk things through. Bring a mental list of issues that you want to understand better. Your goal should be to try to understand what Agile and DevOps need to look like for your organization, given the problems that your organization is trying to solve by adopting Agile or DevOps.

It is essential that executive management evangelize the behavioral changes that they expect, and demonstrate it by behaving in that way themselves. As a great example, leadership should stop shielding themselves: they need to skip levels and talk to people who actually do the work. 

Don’t trust reported status - find out for yourself how things are progressing. Status reports are usually rosier than the reality: traditionally-trained middle management always want to show “on schedule” and “tasks are green” - don’t trust that, and it is not how Agile transformation works anyway. Agile and DevOps transformation are not about being on schedule or completing tasks: they are about people trying Agile and DevOps practices in actual initiatives, and the organizational learning that occurs from those. Status reports are also dumbed down - preventing you from understanding what is really going on.

Visit where the work is done. Make it your business to learn how the work is being done. Engage in long conversations with tech leads and other SMEs without their supervisors present and without powerpoint slides. Ask people what they think, and show that you like being challenged. Ask questions, because you are learning too. Bounce around ideas that are not fully formed - be vulnerable, and allow yourself to be proven wrong. In fact, if someone proves you wrong, reward them with more discussions - they have shown that they know things that you don’t.

Leadership also needs to encourage contained risk - people trying things. Encourage open discussion - demand it, and instigate it. Chime in on slack channels. Give a thumbs up when someone disagrees with you but makes a good point - show that the best ideas win, no matter who they come from, and show that it is safe to challenge current consensus: stand with those who take the risk to do so. Share articles and ask “what do you think?”.

Permit some level of “confusion”. Cultural change will follow.