The Craziest Things with Methods and Frameworks

Ivar Jacobson and Simon Girvan

Ask software developers what they think they are doing and you will get answers remeniscent of the parable of the ‘blind men and an elephant’. A group of blind men who never have experienced an elephant before are asked to tell what they think it is by touching it. Depending on which part of the elephant they touch, they come up with different answers, each convinced they are correct. The moral of the parable is that humans derive their truth from their own limited and subjective experience, and have a tendency to ignore other people’s experiences that may also be true. ?It is only by putting all the perspectives of the truth together that we get the full picture.

With software developers, the answers are similarly colored by the respondents’ experiences of their branch of software. For instance system engineering, user experience or highly regulated industries..

Over the 50+ years of building software, we have never had a uniform opinion of what kind of work it really is. We have zig-zagged between seeing it as an art, a craft or an engineering discipline. Most of us would agree it is a little of all, but how much varies. Similarly, most of us would agree that it is a creative activity, but how much of the work is really creative and how much is ‘no-brain’ work (obeying well-known patterns) varies in the same way.

Thus, over the years the most popular methods have also zig-zagged – some being light and non-prescriptive assuming that the work is mostly creative and an art; others being rich and prescriptive assuming it is mostly routine and engineering. However, the different ways we deal with methods and frameworks demonstrates the immaturity of our business of creating software.

Co-author of this paper, Dr Ivar Jacobson, knows this problem better than most because he was once a Method Guru himself – but has become increasingly frustrated at the damage this causes to our industry and devoted time and energy to diagnosing the problem and proposing a solution.

“Twenty years ago, people saw me as a guru in the software industry. I had been a creator or a co-creator of component-based development, use cases, Rational Unified Process (RUP), Unified Modeling Language (UML) and more. Life was a “bed of roses”!

However, I also felt that having gurus in a science-based industry, social and technical sciences, was unworthy of our discipline. Originally, I felt it was unhealthy, but over time seeing that nothing has changed I feel frustration - it is nothing less than crazy – a word not normal in my vocabulary

Instead of attempting to create a better method, I have focused on recognizing and addressing the dysfunctions in how we work with methods.”

--Dr Ivar Jacobson

In this article, we will discuss 5 of his most dangerous findings – the ‘Craziest Things’[1], and share an insight into how they can be resolved.

1. We are in a methods war for more than 50 years
2. Practices are locked in method prisons
3. Method prisons are controlled by gurus
4. Methods have a lot in common, but have no common ground
5. Methods have an Achilles’ Heel; they are theory only with poor support for putting them into practice

We are using the term method as a synonym for method framework, process, methodology, etc. even if strictly speaking they are different.

Each method is composed of a number of “mini-methods” – such as Scrum, user stories, test-driven design or self-organizing teams - dealing with different and separate aspects of development. We shall use the term practice for these “mini-methods”.

1.0 Methods War

“The term?Methods War?has long?been used and comes from the idea that a community supporting a particular method is competing, sometimes fiercely, with other communities supporting other methods.

Every method has a founder,?and around him or her is a community of people supporting the method with training, coaching and maybe tooling. It works almost like a pyramid scheme – the approach has worked very well for the early adopters, so they become evangelical about it to others. The language used to sell relies on preaching and since the communities are separate,?not collaborating for the best of the industry, they remind us of sects.

If reasonably successful, the founder is a great marketing and sales guy and uses such techniques to win members and clients to his?or?her sect. Very successful founders become?gurus. I became one, so I believe I know how it works.”

--Dr Ivar Jacobson

Competition between product or service companies is not an inherently bad thing. Neither is differentiating your product by using unique, fun or idiosyncratic language and creating communities of evangelists to champion it.

However, there is a difference in competition between “normal products” and “method products”. Methods appeal to your intellect. They are mind changing; sometimes genuinely paradigm shifting. Because the conversion to a method is an emotional and intellectual one, it invokes passion in its followers. Teaching them involves helping the student reject how they currently think and behave; and to embrace something new, different and often uncomfortable. The competition therefore reminds us of preaching and religion; and of religious wars. We also observe this phenomena among other products: for instance Apple and Android phones, or Tesla owners and drivers of non-electric cars.

We saw examples of these wars when SAFe first emerged into the Agile world; we see it where Agile approaches begin to take over from formal Project Management; we see it when internal processes get compared to the latest on-trend commercial method; and we see it when an enthusiastic developer returns from a conference full of bright ideas.

Recognizing that method wars are destructive to our world is the first step. Then, we can work together where we have common ground, and share our expertise where we are unique experts.

2.0 Method Prisons

”Today, the practices within a method are locked into that method—they are in method prisons—and cannot easily be reused in other methods. In fact, to reuse a practice it requires that it be rewritten to fit the homegrown style of the method using it.” ?

--Dr Ivar Jacobson

That ‘method prisons’ are a crazy thing?is not generally understood. After all, most people assume that having all the practices you need bundled into one big method is a benefit, not a problem. To understand why this is a problem, we must first understand how methods are constructed in the first place.

This is what happened with RUP (originally created by Dr Ivar Jacobson and named Objectory), and it is a pattern followed by many other methods:

1.????It begins with the method creator being an expert in some particular practice or practices that other people are interested in. She or he may have invented them or written a paper or a book that becomes successful.

2.????Other people start using the practice(s). But they don’t cover the whole of the development process, so the creator, now taking the role of methodologist, starts to augment the original practice(s) with more. In doing so, the methodologist tends to develop their own way of describing their method – structure, vocabulary, notations, etc. Everything is homegrown. Methods share almost nothing apart from the alphabet .

3.????Since no methodologist is an expert on everything, she or he must incorporate (‘borrow’ or ‘steal’) other methodologist’s practices into her or his own method.

4.????No methodologist with self-respect would do that without ‘improving’ the practice, making it consistent with their other practices and perhaps changing some of the language. But these ’improvements’ are not always improving the practice. Frequently, the original creator of the practice feels the practice has been misunderstood or even damaged.

5.????Any good method will undergo regular improvement and be updated. The same is true of practices. However the two are not usually in sync; big things like methods tend to change relatively slowly even if their contained practices have changed a lot.

6.????As a method evolves, it will tend to grow. As it grows it resembles a monolith and becomes harder to change, harder to comprehend and harder to master. Simply put, it becomes clumsy.

The result of this behavior is that many practices are reduced to merely being part of a single method, limiting their freedom and constricting how they can be applied. They are imprisoned by the method.

Method prisons are not restricted to commercial or marketed methods and frameworks. They also exist within organizations. Organizations commonly evolve their own distinct development approach that isn’t written down, but communicated by behaviors, folklore and word of mouth – “This is how software is developed here”. This makes them very hard to explicitly change – one cannot change something one cannot see. Their practices are caught in their own method prison.?

Even agile methods – which are mostly light in description - are not immune from becoming prisons. They also suffer from the same problem of not supporting reuse, mixing and matching practices etc. and use exclusive language and tightly coupled practices.

It is clear that imprisoning practices is not helpful to our industry.

3.0 Gurus

“Among people interested in software development methods, the term guru has been used for someone being the creator of a method or a framework that is or has been successful. A guru is according to Oxford Languages “an influential teacher or popular expert”. I learnt the term from people calling me a guru because I was a co-creator of RUP.

A guru is usually just an expert on one or a few practices, while the other practices are “borrowed” from other gurus or experts. The original creator of a practice, the expert, may get a pat on the back from the guru borrowing it, but that is usually all. And it is not fair, because the creator has not just created the practice, but also invested both time and money to make it practical and popular. We need experts, but not gurus; it is unworthy of the industry.”??

--Dr Ivar Jacobson

Throughout the evolution of software development methods individuals have become well known and even famous because of their association with particular methods. This persists today – you only need to look at how big software conferences advertise keynote speakers.

Every successful method starts somewhere. A common pattern is the expert (sometimes a few experts), has some new popular ideas, formulates them as practices and composes them with a good selection of other, existing, popular practices, into something new. They begin talking about it, perhaps publishing papers or a book, and there begins to be some energy and excitement around the ideas.

This success means that the composition is received very positively by people in need of a method. The expert is, of course, very knowledgeable on his or her own practices and has a good understanding of the other selected practices. However, that is not enough. He or she makes it highly believable that the method is great, requiring him or her to become a great marketing and salesperson.

Now the expert becomes a guru.?Ivar?together with Philippe Kruchten and Grady Booch were once the?gurus?governing the Unified Process?(and RUP) and realized that this was?a crazy thing - unworthy?of?any industry, particularly?a huge industry?such?as the software industry.???

Now, not only do we have a methods war, we also have a Gurus war.

While it is good that methodologists ‘borrow’ practices from one another, the problem is that a methodologist doesn’t have any really good means to reward a practice creator. Particularly, when he or she has rewritten the practice and created a channel to market the practice, not with the support of the creator. Normally, the only reward is a pat on the shoulder such as a mention in a footnote. The consequences are animosity between top people who together could make a good contribution to the user community.

This sets software apart from other, more established engineering disciplines, that ?rely more on experts than on gurus. There are plenty of experts in specific engineering practices and technologies, but none who would profess to be the expert in the whole of a discipline like renewable energy or broadcast systems. Those fields of engineering don’t have the equivalent of a RUP or SAFe or Disciplined Agile that assume that the same set of practices ought to be applied in similar ways to most projects. Instead, practitioners, teams, and organizations select the established techniques and practices they need.

This makes the practice their first-class citizen. The method is subservient to the practices.

A consequence is that other fields of engineering have no need for a small number of gurus – instead, there are a large number of detailed experts who each has more time to develop her or his craft, advancing their practice further.

The existence of Gurus in in software development is therefore a crazy thing.

4.0 The Absence of a Common Ground

”Every method has some unique practices, but it has a lot more in common with other methods. After all, they all deal with software (or systems) so they should share a lot.

The fact is, what they share is hidden and not explicit, so without deep inspection it seems they share almost nothing, not even the basics such as: What is Software Development? What are Requirements? What are Teams? Somewhat exaggerated, they only share the alphabet! Given we have developed software for more than 50 years, this is unworthy of our industry.”

--Dr Ivar Jacobson

Although all software development methods are concerned with developing software, they usually use different language to describe similar things. Methods often have homegrown vocabulary, presentation, roles and rules. Often, almost everything is special and unique for a method making it very hard to learn more than one method. There are some good reasons for this uniqueness. From a branding and marketing perspective, consistent look and feel is important. Introducing new or esoteric language can grab the attention of the potential customer and generate interest. A quirky style can help a method stand out from its competitors.

Each method creator also selects or designs a unique distribution channel. Some are as simple as being freely available to download. Others are more closed with the method behind a paywall and a growth model that reminds of a pyramid scheme. Early supporters (trainers and coaches) will benefit the most and motivate newcomers.

The absence of an agreed common ground is perhaps the craziest thing in the list of Crazy Things. Thus, let’s get a common ground.

5.0 The Achilles’ Heel of method adoptions

“A method is usually accompanied by books, papers, blogs, websites, slides, certifications, training, which is the theory. Putting that into practice is at best served by coaching, which is hard to get for many reasons, one being that the company culture is not to buy services.

I call this gap between learning and doing the ‘Achilles’ Heel’. The people expected to do the job are more or less left alone when they have to do it. They can go back and look at the theory, but that will soon be forgotten so they risk soon falling back to old behaviour. This is why successful method adoptions,?in the long?run, are at very high risk. That risk must be mitigated, and coaching is clearly not enough. We need to cure?’the Achilles’ Heel of method adoptions’.”

--Dr Ivar Jacobson?

We can describe the Method Adoption loop as two cycles: The Learning Cycle and the Delivery Cycle.

Most methods, even the widely used ones, are made accessible through books, papers, slides, videos, etc. The learning cycle is basically up to the user herself or by being taught in class with a quality certification by the method provider. The user has learned the method theory.

However, when it comes to the delivery cycle –putting the method into practice - the user is on her/his own. So far, the method creators have not provided any help when the users are actually doing the job. We, who create methods, have focused our efforts on developing training, possibly complemented with coaching. However, there are not many good coaches available, and if they are often wasted on routine tasks. This situation was true for RUP, and it is true for the most modern and popular methods of today. It has not changed over the last 20 years. In fact, it has not changed since the beginning of software development.

Thus, the learning cycle - methods in theory – is rather well supported, but the delivery cycle - methods into practice – is on the whole not supported. There is a gap, a big gap, to bridge for the user moving from the learning cycle to the delivery cycle.

That gap is what we call the Achilles’ Heel of Method Adoptions.

?

6.0 Essence – a golden solution to the ”crazy things” and more

In the previous sections we have described five “crazy things” working with methods. There are many more, but these are at the top of our list. The good news is that we believe there is a solution – the OMG standard Essence [1],[2]- that can eliminate or at the least dramatically reduce them.

Essence is simple and elegant; a powerful base upon which to create methods that helps teams build successful products. Of course, it addresses the Crazy Things. However, its real value[3] comes from the many other challenges it addresses, one of which is the ability for a team to select the practices it needs from an open eco-system of practices and compose these practices to a method. Essence works as an eye-opener. Once you 'get under the skin' of Essence, you understand these challenges and their solutions[4].

Essence is a simple concept but, much like Agile before it, to get the best from it requires a shift in mindset and to think differently about how we develop software solutions.

Think of Essence as a description of what is common for all approaches to developing software. A particular method can reuse this common ground as a starting point when describing what is specific, instead of starting from nothing (but the alphabet). Essence identifies 7 important dimensions – called alphas - that need attention for any method to be successful. These alphas are in three areas:

1.????The Customer area– Opportunity and Stakeholders

2.????The Solution area– Requirements and Software System

3.????The Endeavor area– Work, Team and Way of Working.

During the lifetime of an endeavor these alphas move in some coordinated way from state to state. Since Essence is method agnostic, these state transitions are not specified. We say that Essence specify what needs to be progressed. It doesn’t specify how. The how is specified by the practices described using Essence as a base.

By identifying which state each alpha is, Essence can also be used to identify where an endeavor is in its lifetime. Finally, to get the whole picture, every state of every alpha is specified by a checklist expressed in terms of real outcome, not in terms of work done or documents produced.

Essence describes the common ground that we so evidently lack, and it does so in a freely available open standard that anyone can access. This common ground is true for any kind of software endeavor: Agile, waterfall, single team, scaled, anything else you can think of. Plus, Essence is designed to be extended in all possible ways, making it very flexible.

What is surprising is that?if we can eliminate this one specific crazy thing – the lack of a common ground, we more or less directly will eliminate or dramatically reduce the other four crazy things. Therefore, we see Essence as a golden solution to the other crazy things. ?

Let’s see how as we once more look at them one by one:

6.1 The Methods War

The?methods war?will be eliminated, because?there is no more anything worth fighting for. By connecting practices directly to the alphas and the states that they action, they become first-class-citizens, meaning practices are the things that we care about. ?

Methods become just compositions of practices. It becomes easy to modify, add or subtract practices as you please. Since, methods (or frameworks for that matter) can be created simply, we will see the creation of many more methods than we see today, but we won’t make a lot of fuss about them.

One interesting side effect is that we will get an end to branded methods. Instead, practices, the ingredients of the methods will become the center of interest, which is where they belong. And it gives creators of practices a more prominent and natural role in the business of software engineering.

6.2 The Method Prisons

As a consequence,?method prisons[5]?will also disappear, because practices are freed from their methods and accessible from an eco-system – global, local or both. With proper tooling a team can browse and discover which practices are available; they can explore the ones they find interesting, adapt them to the specific needs of the team, and adopt them.

It may be that a team composes its method with the same practices as the guru methodologist would have chosen, but it doesn’t have to. The team can make their selection based on their specific needs. As time goes by, and the team inspects and adapts , it can upgrade its method with new or updated practices.

Once we have these practices, we require some way of organizing and improving them. We need some kind of practice library and way of describing practices independent of its source. Essence includes a language that describes practices in an open, sharable and standard way. Much like the Essence kernel elements describe the common ground of software development, the Essence language is the common ground to describing practices and their elements.?

Many people have suggested that if they adopt Essence they will get into the Essence method prison instead; that Essence is a ‘Wolf in sheep’s clothing’ trying to replace existing methods with a new, better approach that is ultimately the same trap.

Here is why that is not true. Today, every method not being essentialized is a prison for its practices. If a method being essentialized would be in an Essence prison, it would mean that there would be other common grounds on top of which practices could be described. Not impossible of course, but what would it take? Recall Essence has had the scrutiny of an International standards body. It is stable, and it doesn’t change unless clearly motivated.

Moreover, it has been designed to be easily extended.

However, it will change. One change being planned is to make the Essence kernel more universal, with the current, software centric kernel being an extension of this universal Essence. It is a relatively small change. It is hard to see what it would take to abandon Essence with its ongoing improvements keeping compatibility backwards. As an aside, Essence is, for example with Scrum and Scrum@Scale, already applied for many other kinds of systems than software systems.

Essence doesn’t replace methods because the Essence kernel on its own just tells you where you are and what you need to do – it doesn’t tell you how to do it. This may be enough for competent teams, but most teams need more guidance. That guidance is provided by practices defined on top of it, thus using Essence as a platform.

6.3 Gurus

The investment the world makes in developing software is huge, maybe far larger than the development costs of any other industry. With such a prevalent and important discipline, why would we need methodology sales people - gurus? That implies that we need to be told what to do and that a small number of people know this vast industry well enough to be able to tell us. It belies the fact that software is a creative industry with talented people who are often required to be more like artists than assembly line workers.

Basing software development on Essence - with many practice experts building practices on the foundations of the Essence kernel - means that there is no need for gurus. And no need for the software community to seek a guru to look up to.

6.4 The Achilles’ Heel of Method Adoptions

Methods are only Theory!?Put the theory into practice by eliminating the gap between learning and delivering. We call this gap the Achilles’ Heel of method adoptions.

Essence and essentialized methods are actionable. With proper tools, such as Essence in Practice TeamSpace?, we can give guidance to the people having to do the job as they work. They don’t need to go back to the study material to find out what they should do or reply on memory. Essence presents guidance and checklists close to hand and and as physical cards, making the guidance simple to act upon or even just be inspired by. This simple form factor also enables continuous improvements through Practice Retrospectives and make a practice easier to learn, as Jeff Sutherland has discovered using the Scrum Essence Cards in his classes.

Having practice guidance close to hand makes it easier for the user to perform the practice. Thus, the user can focus her attention more on the creative elements of the practice or on a better way of working to improve the practice. In this way, the learning loop is traversed alongside the delivery loop – as it should be.

Eureka, we dare say that?Essence puts Methods into Practice.

7.0 Conclusion

A common ground for all methods and frameworks will eliminate or dramatically reduce the crazy things in working with methods. Essence is such a thing and it is an international standard. Why don’t we all use it?

There are many reasons. There are of course commercial reasons. Companies have invested in ways of working that belong to the old school, suffering from the crazy things. Another reason is that it is not trivial to first “get under the skin” of Essence. Our world has never had anything like a common ground before so you will have to spend a couple of days to transform your knowledge and experience from today to an Essence based situation. There are some subtle but important mindset shifts necessary to get full value from Essence, much like there are with Agile values and principles – and look how long it took for Agile to become established.

However, we have never seen as much interest for Essence as today…the curve is growing exponentially, with involvement from industry and academia. ?We are now working with?many?thought leaders on?essentializing their practices and several large companies are now on the path do adopt Essence as a platform for their eco-system of practices. We feel very close to reaching the tipping point!

As you can readily understand, the success of Essence is not coming from eliminating the crazy things. It comes from the value propositions that adoption of Essence gives to organizations, teams, individuals, thought-leaders and many more.

These have been described in many articles. ?

Let us just mention a few personas and the values they get through using Essence:

·?????The team will benefit greatly as it is easier to learn, modify and maybe most importantly use practices. They can be empowered to select the practices that work for them, and not have to work the same way as everyone else.

·?????The coach (scrum master, method expert) appreciates that it will be easier to learn many different methods, easier to get updates on practices as the organization learns more, and easier to help the teams to help themselves. It is straightforward to mix and match practices from various sources to create an optimum blend for each team. The simplicity of Essence helps coach practice improvement and teach practices to new teams.

·?????Program Governance values a single common ground and minimum viable governance that provides leading indicators of progress (or problems) without constraining all of teams and products to using the same method. More empowered and accountable teams tend to result in higher quality work.

·?????The executive wants Essence because it helps in moving development from primarily being a craft to primarily being an engineering discipline. It helps in building a forever learning organisation, and it provides improved governance of ongoing projects and programs.

These are just a few examples of the values Essence has to offer. Almost everyone concerned by software will experience their work in a new, different and better way.To achieve this we have developed tools implementing the Essence standard and langauge.

Essence in Practice WorkBench? ?supports the static aspects of Essence, for example to create practices, to compose practices and to print practice cards.

Essence in Practice TeamSpace? deals with the dynamics, to activate the practices, customize team practices and checklists, and track the state of their endeavors.

Finally, industry as a whole benefits from having for the first time a common ground of elements to use and a common language to express practices, all focused on the essentials.

This makes it easier to compare, mix and improve practices from various sources and make them available to others. This will accelerate the learning and sharing cycles across organizations and liberate the learning and knowledge from around the world.

Academia gains the same structure with Essence to help with research and share findings, while universities are increasingly using Essence to form the basis for their teaching. Students of the future will hit the ground running in their new jobs, holistically understanding what is important, and speaking and thinking with the same common language.

?

?

?

?

?

?

?

?

?

[1] If you are interested in?discussing gurus and the other ‘crazy things’ further, please join our LinkedIn community Essence for Agility | Groups | LinkedIn-?The Craziest Things in Working with Methods and Frameworks | Groups | LinkedIn.

References

[1] “Essence – Kernel and Language for Software Engineering Methods”. The OMG standard specification. ?https://www.omg.org/spec/Essence/1.2/PDF

[2] “The Essence of Software Engineering”, CACM, https://essence.ivarjacobson.com/sites/default/files/field_iji_file/article/essenceofsoftwareengineering.pdf

[3] “Value Proposition of Essence”, https://www.dhirubhai.net/pulse/value-proposition-essence-ivar-jacobson/

[4] “Learn More about Essence”, at Ivar Jacobson International

[5] “Tear down the method prisons! Set free the practices!”, ACM Queue, 2018.