Agile Core Ideas – Something Is Not Quite Right

It’s heresy to challenge the Agile Manifesto. But it is necessary. How else are we to refine our thinking?

We need to remember that the Manifesto is not a deific work that is perfect by construction. It is composed of four value statements (which are copyrighted) that were composed over the course of a weekend. The principles were added over email later by some of the participants (they are not copyrighted).

The values are pretty good, but they are far from perfect. The principles are not bad either, but they also have problems.

When I refer to the “Manifesto”, I am referring to the four values. To refer to the principles, which I want to emphasize were added later by a subset of the group that wrote the four values, I will specifically say that I am referring to the principles.

What’s Good

First of all, the Agile Manifesto is insightful. One of the most insightful things about it is that it does not take a position on anything.

That is, the values say very explicitly that there is value in all of the things that they identify:

1. Individuals
2. Interactions (between individuals)
3. Processes
4. Tools
5. Working software
6. Comprehensive documentation
7. Customer collaboration
8. Contract negotiation
9. Responding to change
10. Following a plan

The Manifesto expresses a leaning toward some of these over others, but it says clearly that all of these have their place, depending on the situation.

Those leanings are insightful. For example, they prefer “Individuals and interactions” over “processes and tools”. They are not saying that processes or tools do not matter – they certainly do matter. But the Manifesto’s authors are saying that most (not all) of the time, individuals and interactions matter even more.

But what do they really intend? Is it that individuals and interactions matter more than processes and tools 99.99% of the time, or more like 51% of the time? And what kinds of processes? The term “process” is very general: it can be anything from a set of roles and defined activities (“events”) to a very precisely defined set of steps. The term “process” literally means “something going on”. In my Operations Research studies, the term was used to describe any activity that was being studied.

Very broadly, the Manifesto is not clear, and as a set of values, it cannot be: a set of values is not enough to provide guidance on what one should do. Guidance requires lengthy narratives. Guidance is wisdom: it is contextual, and it is something that one develops over time, through experience, learning, and reflection.

Therefore, the Manifesto cannot be seen as guidance: it is at best a set of reminders of things to think about. It does not provide any insight about the tradeoffs between “individuals and interactions” and “processes and tools” – so as guidance, it is useless. It is just a cheat sheet for people who already internalize the ideas.

What’s Not Good

We did not say that being a cheat sheet is bad. Cheat sheets are useful. What is not good is when people turn to the Manifesto and say, “It says this, therefore we should do that”. That is a misuse of the Manifesto.

Again, the Manifesto does not contain guidance. As such, it is not actionable. It is not a guide. It is not a north star. It is only a set of reminders for people who already know how to find their way – reminders like “birds tend to fly southward as the weather gets cold”.

That does not say “Follow the birds”, because birds fly in all directions during all seasons, but as winter approaches they tend to be seen flying south – how often is not said. But to someone who has experience watching bird behavior, it is a reminder that the direction of birds can be a tool for figuring out what direction is southward; but they know better than to treat every bird’s direction as an instruction for which way to go.

Essentially, the Manifesto’s values are a set of bumper stickers. Here are some bumper stickers:

• Don’t worry – be happy!
• Make love, not war.
• Be circular – recycle.

The first one, “Don’t worry – be happy”, means something deep and complex to people who have lived life for awhile. It cannot be understood well by someone who is young and has not. They will interpret it literally, to mean, “Never worry, always be happy”. But that’s not the intention of the bumper sticker.

The same applies to the other bumper stickers. They are reminders for the wise. They don’t give the non-wise actionable guidance.

Neither does the Manifesto.

Some Things It Got Wrong

Some of the values are kind of confusing, and possibly downright misleading.

For example,

Working software over comprehensive documentation        

That’s not quite right, because it really should emphasize outcomes over working software. Working software is good, but calling that out implies that it is something to focus on. The context of the statement, that it is “over contract negotiation”, implies that it is about how one determines success. But “working software” is an untrustworthy? leading indicator of success: a trailing indicator would be better. Actual outcomes would be better.

The other values are pretty good, so I’ll leave them be. But I will call attention to what is missing.

For one thing, the values mention software, but they do not mention data. Why not? Data and software go together: mentioning one without the other seems imbalanced. We really should expect to see a value pertaining to data, such as “Data understanding over data disorganization”, or at least something to remind us that an organization’s data is critically important for any system development undertaking – particularly to one’s ability to leverage the data produced by the system in new ways down the road.

Another omission is about leadership. The first value,

Individuals and interactions over processes and tools        

says that the interactions between people are really important. But how so? Human behavior is a very complex and multifaceted subject. I’ll turn to the Principles now, because there is a hint there.

It is well known that at least one of the authors of the Agile Manifesto, Ken Schwaber, was very “anti-management”. Perhaps that can explain the principle that reads,

The best architectures, requirements, and designs emerge from
self-organizing teams.        

This principle – even though it is not part of the Manifesto proper, has had a huge influence on the Agile community, and in this author’s opinion is one of the greatest sources of confusion and dysfunction.

It is well known in the field of behavioral psychology that groups of people do not self-organize well, generally speaking. What tends to happen also occurs with chimpanzees: an alpha emerges.

Thus, the way that a group self-organizes is that someone rises up as the group’s leader. This is the tribal lead, and some psychologists believe that tribal behavior is hard-wired into humans.

Among chimpanzees there are two ways that a group member becomes the alpha. One way is to physically dominate. The other way is far more common: the aspiring alpha steals from some members and gives those stolen things to his (it is always a male) friends. An inner circle forms around the aspiring alpha, and that inner circle becomes a subgroup within the overall group. If the power of that subgroup is sufficient to dominate, then the aspiring alpha becomes the overall group’s alpha.

The same thing happens with human groups. The psychological theory known as Leader-Member Exchange (LMX) theory explains it: a leader emerges, and an inner circle of loyalists forms around that person. The rest of the group go along with it.

It is not egalitarian. It is not fair. It is not based on who the best leader is: it is based on who is able to influence others enough to create an inner circle.?In fact, it has been found that people tend to choose tall people as leaders – what does that say?

In her famous essay “The Tyranny of Structurelessness”, women’s movement pioneer Jo Freeman wrote,

“…there is no such thing as a structureless group. Any group of people of whatever nature that comes together for any length of time for any purpose will inevitably structure itself in some fashion…

“…structurelessness becomes a way of masking power, and within the women's movement is usually most strongly advocated by those who are the most powerful (whether they are conscious of their power or not). As long as the structure of the group is informal, the rules of how decisions are made are known only to a few and awareness of power is limited to those who know the rules. Those who do not know the rules and are not chosen for initiation must remain in confusion, or suffer from paranoid delusions that something is happening of which they are not quite aware.”

This is a warning: those who strongly advocate for self organization are often the ones who see themselves flourishing in that situation: the ones who are outspoken and good at influencing others. They know that they will become the leaders.

What the Manifesto’s values should have said about this was something like what Agile 2 – a much more diligent effort to characterize organizational agility – says in one of its six values:

We value all of these things, and strive to balance or combine them for each situation:

…Individual empowerment and good leadership.

…

and the principle,

10.2 Principle: Provide leadership who can both empower individuals and teams, and set direction.

In fact, Agile 2 has no fewer than nine principles pertaining to leadership, because leadership is a complex topic. It is not possible to write a single statement that captures the important reminders.

This is the biggest thing that the Agile Manifesto got wrong: its incorrect emphasis on self-organization, and its failure to say anything explicitly about leadership. That’s like writing a list of reminders about sailing and forgetting to mention the sea.

Self-organization is an outcome: If an organization has a healthy (“Constructive”) culture, and its members have high EQ, then one tends to see teams that self organize and that self-organize well. But that is unusual, and it is certainly not a starting point for organizations in general. Indeed, telling a typical organization to have its teams self-organize is like a doctor telling a patient to just “be healthy” – but good health is an outcome, not a prescription.

It’s like the Bob Newhart skit where he plays a psychiatrist, and when his patients complain about their problematic behaviors he tells them to “just stop it”. “Just self organize” is like that: it’s like telling your kids to “just be like an adult” and expecting that they will. Forget about all of the learning and maturity needed to get there. Indeed a lot of adults don’t even act like “adults”.

Conclusion

Articles that critique elements of the Agile movement always generate a small percentage of antagonistic responses from “loyalists” who see the Manifesto as an ideological document that they have to defend, kind of like the way that North Korean technocrats need to show fealty to their leader by clapping at everything he says.

Fortunately, most people are not like that. We need to critique things in order to progress. Critiquing something does not mean that it is all bad, or that it was not progress. The Manifesto was a rejection of some very toxic things that had been happening in the IT field during the 1990s, a period which I like to call the “methodology craze”. The Manifesto said “no” loud and clear to those toxicities. That was a good thing. But the Manifesto is far from perfect. It is not a guiding document: one cannot parse its words for hidden meaning. One needs to view the Manifesto as a set of imperfect reminders, and find one’s own meanings.