A thought about the difficulty of changing our mindset

Here is an evolution of this post I put up earlier. It is an update of some thoughts about why it takes so long for some individuals, teams and organizations to adopt a lean-agile mind-set and get real benefit. After working with a number of process improvement methodologies and frameworks over the years, it occurs to me that there is a unique challenge for those trying to adopt lean-agile methodologies or frameworks. The difficult part is not following the processes and practices as these are well defined and simple enough. However just following the practices and processes only leads to the team "doing" agile and not to the team "being" agile. Being agile is where the benefit is.

"Being" agile means everyone thinks and makes decisions using lean-agile principles and values all the time rather than just following the standard agile processes underpinned by old-world waterfall thinking. Many teams try to just 'do' agile and fall short of realizing the real tangible benefits. What's more is they are often not aware they are not achieving the benefits because they have no clear way of comparing current 'agile' performance to past waterfall performance.

I propose the difficult part is in fact the daily decision making process used by team members. They are trying (often very hard spending many late hours) to get the benefits promised by adopting a lean-agile approach but this is being undermined by using old waterfall thinking. This usually manifests itself in terms of how work is structured, planned and delivered. Especially with new teams (although I have seen this in more experienced teams too), it seems quite common that they struggle to adopt principles such as limiting batch sizes, using systems thinking, limiting WiP, adopting economic frameworks etc.. Work items remain "waterfalled" in stages with horizontal slices rather than being planned and delivered in vertical slices. It is common in new teams to work in sprints but only build once a week which undermines the purpose of fast feedback loops. The work tends to still be defined and delivered in big batches rather than small increments while paying lip service to time-boxes e.g. develop in sprint 1, test in sprint 2 and release in sprint 3. Additionally there is little synchronization and cadence across teams so effective communication, alignment with the business strategy and innovation suffers. There are many more symptoms but these are just a few common ones.

I find that team members often don't have complete confidence or don't know how to make their own daily decisions using lean-agile principles and values. Furthermore they may not even realize that these daily 'small' decisions really have a big impact on the ability of the team to improve the flow of work. Individuals often start out trying to work in a lean-agile way but become bogged down in uncertainty when their colleagues don't get it and are still making decisions in a waterfall way. It is difficult to be agile on your own. Structural barriers in the organization may also be impeding these practices e.g. waterfall security and compliance processes imposed on a team trying to be agile. Collectively these decisions can accumulate to make it difficult to foster an agile culture and make it very difficult to consistently perform in an ‘agile’ way irrespective of how many so called agile practices one follows.

No other methodology or framework seems to have this intangible concept of adopting a particular mindset built into its success criteria, for example; I would argue that Six Sigma is rule and maths based, CMMi is process and governance based, ITIL is process and governance based. None of these requires a mind-set change to be effective because the decisions that teams need to make are driven by easy to follow and visible processes, procedures and rules. They in fact only require a waterfall mindset to be operationalised.

I see time and again people commenting on why it is so difficult to adopt an agile mind-set and how to change culture and behavior. It is the subject on virtually every forum, blog, community of practice and training course. Added to that coaches and consultants are always saying “…limit WiP… control batch sizes… start the job with the highest cost of delay first…” and of course “… you should adopt an agile mind-set…” This is fine for the classroom or the certification exams but it turns out that it is not easy to put into practice in a way which results in predictable and stable agility across a team or an organization. There is a lot of research on decision making and plenty of training on what patterns to apply to ensure work flows through the team as fast as possible. The question is how to bring these together for new teams in a manner that they can change from the old ways.

People might know what to do if asked in an exam but be unable to find a way to act on that knowledge in the day job. I propose that if individuals find it difficult to make decisions and act effectively in an agile way e.g. to limit WiP and control batch sizes, then a team or indeed a whole company will find it much more difficult to change how they work in a sustainable manner. After all, if we want a soldier to behave completely differently from a civilian in order to maximize the utility of the regiment, then numerous repetitive drills are used to ensure the decision making by the soldier is instinctive and in support of the broader goal. We don’t just give them a book to read and expect them to do it for real after passing an exam. I am not saying repetition is the only thing missing but it seems to be one of the most obvious. After all we do see people getting better at it the more they try it and if they are willing to learn from their mistakes (or acknowledge their mistakes). However I have also come across teams who have been trying this for years and not got it.

Remember that we have been trained from very young to use phase gates (check with the teacher at each step, check with parents at each step), big batches (study all semester to write a big exam at the end) and so on. It is not simple to change how we think without repetition, feedback and a willingness to learn. Many of the training courses today simply provide the knowledge required to pass a certification exam rather than change our behavior. I suppose it is not the job of a course instructor to change one’s behavior but then whose job is it?

Is it up to the company to continue that training in a specific way after the individuals have been given the academic knowledge? Or is it up to individuals to realize this mind-set change themselves in the interests of the greater good? How will we know when the individual and team is going to be effective and how long that will take?