The Practice of Being Agile

Occasionally, I read or hear something that frustrates and – sometimes - even angers me. That happened recently, when I read yet another article on the state of agile today and why all that matters is the right mindset. Rather than let it annoy me, however, I decided to write this article and explain what I think we should do. Why mindset, while important, is not enough, and what a world looks like when practices and mindset work together.

?

There is a lot of talk about what Agile is and what it is not. It seems as if the most common thread is that Agile is not a methodology or a set of practices. It is a mindset, about principles and values. There is too much focus on adopting practices of technical, business or human nature, rather than focussing on the mindset. I call these 'hard practices' - things like PI planning, Agile estimation, retrospectives, continuous deployment, or test-driven development. While these can all support an Agile mindset, they can also be applied in a very non-Agile way. So, we must also teach what we could call 'soft' practices – those very important skills and patterns more aligned with the Agile mindset such as intent-based leadership, psychological safety, evolutionary change, or continuous exploration.

Many experts refer to Agile as being adaptive to change. Of course, this is the natural meaning of the word Agile. Indeed, the Agile founders did consider the word “Adaptive” as one candidate for the name of what became the Agile manifesto. However, many so-called Agile practices are not easy to adapt. Once they are described they are hard to change and keep updated as teams or organizations learn more. The practices themselves are not Agile.

?

Getting the right mindset and being adaptive to change is a great and necessary start, but is it enough?

When you adopt Agile, it is critical to adopt the Agile mindset, principles and values from the very start. Otherwise, it will be much harder to take the next step: to adopt Agile practices in a way that will actually deliver new business benefits. Thus, if you are serious about implementing Agile, you will need to take the next step. You need concrete guidelines in the form of practices (or patterns). All the successful methods, such as SAFe, DA, LeSS, Scrum at Scale, and their practices are described one way or the other. Oral lore preserved by storytellers may work for small teams, but it doesn’t scale, and it is too dependent on key people for any serious business. Thus, we need some descriptions, but let’s make it clear. We don’t want any massive documentation as was advocated before Agile. We don’t want “fat and flabby” documentation, which unfortunately didn’t die with Agile, but survived in some of the most popular frameworks. We want super light, super effective descriptions only - to learn, compare, measure, adopt, and use them.

However, that is not all we want. We also want to eliminate the craziness we live with. In 2010, 40 thought leaders, 15 large corporations and equally many academic institutions, and more than 2,000 supporters signed a call for action, the first sentence saying: “Software engineering is gravely hampered today by immature practices”. The call for action had good motivations, but as time has passed, we have even stronger arguments, such as these “crazy things” (please read more on these “crazy things” here https://www.dhirubhai.net/pulse/craziest-things-methods-frameworks-ivar-jacobson/ ):

1.????We look like the fashion industry, easily falling in love with new stuff and throwing away old, but still perfectly good stuff.

2.????We have method wars which reminds me of religious wars: if you don’t think like me, we are at war.?

3.????We have method prisons (practices described in the context of one method can’t be used in another method without redescribing it).

4.????We listen to gurus (methodology salespeople) more than we listen to experts.

5.????We have no common ground between our methods and practices. I sometimes like to describe this in an exaggerated way, by saying that the only thing methods seem to share is the English alphabet! They are like isolated islands with no bridges between them.

6.????Industry practice is split from academic research. Of all the huge investments in academic research, how much comes back to the industry?

And many more things which many people have written about in papers and articles since 2010.

Thinking back, these crazy things are not new; they have been around for 50+ years. It is horrible that an industry as huge and important as the software industry suffers from crazy things like these. And for so long. It is unintelligible that very few people speak out about it, but instead just let it continue. How come? Surely there must be a reason?

I am fed up with moaning. Let us act. IMAGINE if we could dramatically change how we work with methods and practices.

?

IMAGINE if teams could get these “killer” use cases (I use the term ”killer use cases” to denote new use cases that have not previously been available and have dramatic value.)

Select their own method. As a methodologist I like to name things; when a team has a repeatable approach to doing something with a specific purpose in mind, I call it a 'practice'. But teams usually face several challenges and so need several practices. I call a composition of several practices a 'method'. Teams have unique problems, preferences, experiences, and context. Applying the same method to all teams will almost always result in a poor fit, resulting in unhappy teams and a poorer outcome. For example, the best method for a team working with severe real time requirements is quite different from that needed by a team working on big data. Yet they may both be part of the same larger solution and constrained to have to work the same way.

Bridge the gap between learning and delivery. This gap, which I call The Achilles’ Heel of Method Adoptions, has been a big hurdle as long as we have worked with methods. Methods support teams during the learning cycle only, not during the delivery cycle. Thus, when a developer has learnt a practice, he or she has usually severe difficulties in applying it. Historically, the only way to combat this has been to recruit consultants (or internal experts), but few are good and those that are good have to spend most of their time telling teams what they could be told better directly from the practice.

Activate the methods. Existing methods offer passive user experiences usually based on slides, handbooks, web sites, videos, etc. They are also not sensitive to the state of the work. The UXs were usually designed many years ago when the methods were originally created. We want active methods, supporting the teams both when they learn and when they deliver. ?Modern facilitation techniques such as game playing, and progression awareness make a huge difference.

Gamify software development. Gaming is not new in software engineering. Games motivate, stimulate, and engage the teams. The problem with existing gaming techniques is that they rely on a language that must be understood intuitively. They use nice graphics, but the meaning of all the graphics is not explained; they lack defined semantics. Consequently, it is only possible to practically create a handful of games. We need a language with deep semantics so we can define lots of practical games. But the language must be very simple and easy to learn and use.

And IMAGINE if organizations could get these “killer use cases”:

Create an ecosystem of good practices, in which the practices are updated – both variants and versions, by its founders or by other experts based on their up-to-date experience. Teams and organizations can access this ecosystem and get the latest and greatest practices working for them. They can compose the selected practices into a complete working method and easily upgrade that method as more experience is won – internally or globally.

Not having such an eco-system is a real problem. As an example, use cases were first widely published in 1992. Most organizations applying use cases adopted this version and were unaware of the many updates that happened afterwards. In 2005 we published Use Case 2.0 which essentially included user stories, but very few users of the original version became aware of the later version. There was no way for us to reach them. The situation is the same for other practices, which is an enormous loss of knowledge. An ecosystem of practices would be of immense value.

Create a systematic way of describing practices. The language can be simple and highly intuitive, but it needs to have precise semantics. Then we can achieve the needed systematization.?We can provide use cases that make working with methods an engineering activity. We can create an ecosystem of practices from which a team can select the practices they need and compose them into a working method, and more.

Focus on the essentials in the description of every practice – don’t try to make them complete. The essentials, being say, 5% of what an expert knows, would be enough to participate and engage in conversations with team members related to the practice. ?That would significantly remove the “fat and flabbiness” we still experience even today with documenting Agile practices. The more complete description is still available for the rare occasions when it needs to be referred to.

Create a systematic learning organization. An ecosystem of practices will become an asset of great value to a company. This is where teams and individuals in different parts of the organization, possibly working with different products can capture the collective experience of its people. Without such an ecosystem there will be many dialects of the same practice. Teams won’t learn from other teams working with different products; there is no systematic path to learn from teams working with other products. Moving from one product organization to another will require relearning a lot of what you already know. With an ecosystem you can always find the latest and greatest on any practice within the organization.

Move from software development primarily being a craft to primarily being an engineering discipline.

It is easy to imagine that executives or the CTO in a company heavily dependent on software would applaud any effort leading to software development being more of an engineering discipline. The reality is that in most organizations, having adopted Agile principles and values, software development is essentially a craft. Where possible, practices have been adopted that make work partially streamlined. The developers may be content with that, but we all know that craftwork is expensive, and routine work results in poor quality.

Thus, we live in a world of contradictions. Executives want more engineering, but how; they have no means to enforce it. The developers love to be craftsmen, and not part of a process forced upon them. The power is with the developers who produce value, so they will win the match.

How can we bridge this contradiction? I think we can do it by reducing the no-brain, routine work and increasing the creative time developers can spend on solving real problems.

There are guesstimates that 80% of the work when we develop software is no-brain work; the developers follow patterns known to them, meaning the creative part would then be just 20%. There are exceptions of course, but where the majority of development occurs (in companies with a lot of commodity business) this may be an accurate guess.

Now, how could we imagine really making this move?

I think there are some interesting ways:

Thanks to the systemization of how we work with methods, we can apply modern AI techniques such as machine Learning. Mr. Satya Nadella, the CEO of?Microsoft, has reportedly said that all Microsoft products will be supported by techniques like ChatGPT. In other words, Mr. Nadella doesn’t think AI will replace any existing Microsoft product, just make them significantly better. Similarly, a practice eco-system is a prerequisite for practically applying GPT-like techniques. Every developer will be supported by a co-pilot when writing practices, when selecting practices and creating methods, when applying activities or producing work products, etc.

?

Is this just a dream? No

Hundreds of experts have been working for more than a decade to realize this dream; the dream to get rid of the "crazy things" and to get the "killer" use cases for teams and organizations. Some have worked on it for two decades. The realization is Essence, the international standard, and its use cases. (‘Learn More about Essence’ https://www.dhirubhai.net/pulse/learn-more-essence-ivar-jacobson-1f/ ). Essence has been designed by experts of not just software engineering but also of computer science. Therefore, Essence is a simple language, but it is mathematically defined (using denotational semantics) which allows us to develop advanced tools to make software development more of an engineering discipline.

Essence is getting widely adopted. Many organizations are now, while still committed to Agile, giving up their popular Agile framework. They think they have learnt something, quite a lot even. The academics are adopting Essence in large groups. Furthermore, more and more methodologists describe their existing work based on Essence to make it significantly easier to adopt.

Really dreaming on a fabulous future: we need more science, more theories. There are many mini theories, such as Constantine’s ‘Cohesion and Coupling’ or Meyer’s ‘Design by Contract’, and we need many more of them. Ideally, we would of course love to get a general theory like Maxwell’s Equations in Electricity or The Theory of Organizational Structure in sociology. I have good reasons to believe we will get one.

We want ‘A General Theory in Software Engineering’, and there is hope. Prof Pontus Jonson, KTH, says in “The Essentials of Modern Software Engineering” https://amzn.to/398wL9i :

???????“Essence takes the first step by proposing a coherent, general, descriptive theory of software engineering (i.e., a language of software engineering).”

???????“As a descriptive theory, the Essence can be used to describe and facilitate discussion of future predictive theories of software engineering.”

And to be complete we need both the descriptive and the predictive parts of the theory. We welcome top researchers to contribute and make a real footprint to the field of software engineering.

The future is mind breaking!

Get started with Essence here: https://www.ivarjacobson.com/essence-explained-agile-tools . You can also join the Essence for Agility MeetUp https://www.meetup.com/essence-for-agility/?_cookie-check=k_i3V_G7sQiIRSwA .

?