WHY SHOULD TEAMS MASTER COMPLEX ADAPTIVE SYSTEMS?

What are Complex Adaptive Systems?

Simply put, complex adaptive systems (CAS) is a science that explains how a group of diverse, autonomous, and connected parts reach a common goal using external feedbacks to adapt.

These systems are collections of parts that are integrated to reach a certain goal using input and producing outputs. What makes them complex is the fact that although we understand how an element or agent operates, due to their diversity, autonomy, and connectivity we cannot predict how the system will behave. Complex systems transform into complex adaptive systems when the agents of the system continuously find ways to adapt to the external and internal environment.

CAS is a relatively new field in science that started in the mid-1980s by the Santa Fe Institute in New Mexico. CAS is rooted in Chaos theory, system theory, and cybernetics. It is a multidisciplinary field that brings knowledge and experience from many fields. It’s a fascinating field that can positively impact management and leadership.

What are the relations between CAS and Agile?

Agile was a response to a crisis that the software industry experienced in the early 1990s. Personal computers started to become common in organizations and the demand for new software increased. Software development was a long process that took an average of 3 years to complete a project, at which the requirements became absolute because the needs of the organizations changed before the product was completed. The problem was that the software industry utilized practices from the engineering industry. Engineers dealt with complicated problems, while software dealt with complex problems. The difference between complex and complicated is that complex is hard or impossible to predict the behavior (an earthquake) as complicated refers to an expected behavior, yet hard to understand how it works (a watch).

The agile movement tried to deal with complexity from multiple directions. Too many computers to run the code, too many diverse and autonomous people with different requirements, and complex business environments that forced the needs of people to change continuously. As the leader of the information revolution, the software industry is considered a pioneer of complexity.

Because of this complexity, you shouldn’t be a surprise to see this quote from the father of Scrum “The key part of what makes Scrum work is “complex adaptive systems” and you need some understanding of this. Also, you need to understand systems theory and how to optimize a system as a whole.” ~ Jeff Sutherland (Interview: Dr Jeff Sutherland, father of Scrum). Actually, you will find that CAS not only influenced Scrum, but it also influenced the Agile movement and lean manufacturing. The Ideas of Chaos theory and CAS influenced many industries, especially after the success of the Toyota Production System at NUMMI.

What else can we get from CAS?

The science of CAS defined several attributes that describe how those system survive and thrive. You should have a good understanding of these attributes. Knowing them will give you a better understanding of one of the agile pillars and ability to leverage them within your daily work. Parts of them will be familiar to you if you’re agile practitioners, other parts will be new and sound radical, but they are not more radical than the one that you already fully or partly leveraging. 

The first and probably the most distinctive attribute is distributed control or no central control. No one cell controls our brain, nor one agent controls the market. Yet, systems are working to reach their goals and to continuously progress. CAS doesn’t say anything about management, but they are clear about the disadvantage of central control in complexity. 

Complex adaptive systems are self-organized. No external authority organizes the system or parts of the system. Self-organization is not just performing work without waiting for a manager to assign it, it’s creating order in a system from the interaction of agents. This can also be seen as a bottom up approach. Self-organization is not a one-time event and is used every time it needs to deal with a new challenge. Neurons in our brains are self-organizing all the time. 

Emergent is the ability of elements in the system to create new patterns which makes the whole greater than the sum of its parts. It is based on simple rules performed by many agents that create a new property. Emergent properties are one of the competitive advantages of CAS. A flock of birds is a classical example of emergent properties. 

CAS govern themselves using feedback loops. Elements in the system have feedback loops and get direct feedback if their actions achieve the right results. If not, the elements can self-organize, create a new emergent property and try again until the feedback is positive or the system dies. 

In CAS, agents can be specialized in certain tasks. We can find a mix of competition and collaboration between agents. But, there aren’t any silos. Information is not kept intentional in one part of the system, and although parts are specialized, they all take part together to reach the system goal. Similar to the subsystems in your body. 

Let’s look at other attributes that you might be less familiar with, but they have a part in the success of CAS. Co-evolution is the concept that once one element in the system evolve because of feedback, it will slowly cause most or all the elements in the system to evolve as well. The change won’t necessarily be the same, but there will be a change. This is a continuous attempt to find the best way to deal with current challenges. 

Sensitivity to the initial condition is an attribute taken from chaos theory. In means that tiny changes to the condition that start any activity will have an unexpected impact on the system after the activity will take place for a while. Because of the nonlinearity of the system, it is impossible to predict that future state, but it is possible to go backward and understand how the system behaved. It means that determinism (based on historical and current data the future can be determined) and linearity (a direct line between cause and impact) are not applicable to complex systems.

Equilibrium is death for complex adaptive systems. Complex adaptive systems need to push themselves all the time far away from Equilibrium. It means that CAS needs a combination of chaos and order to survive and thrive. CAS need to move between stability and instability, competition and cooperation, and order and disorder. State of paradox is the preferred state of CAS. 

How to implement CAS into your operating principles?

Now, let’s take a look at how these theories work in the real world. I was fortunate enough to have the opportunity to implement the principles of CAS mentioned above for 5 years for an IT group of 60 personal for a mid-size company. the outcome was very favorable with metrics that were significantly above average. 

I won’t go through all the change management process we followed before and during the implementation. They can fill several posts by themselves. In this post I’ll focus on the technical implementation.

To make the implementation easier and to be able to make sure that basic assumptions are working in reality, the first thing we did was to split CAS principles into 3 cores:

·      60% of the team focused on the first core (Distributed Control, Emergent, and No Silos.)

·      30% of the team focused on the second core (Self-Organization, Feedback Loops, and Far from Equilibria.)

·      10% of the team focused on the third core (Dependency on Initial Conditions, Co-Evolution and State of Paradox.)

Each core was implemented by several groups. Based on the initial feedback we augmented the attributes and increase implemented group until all the people in the group operate following the adjusted CAS attributes,

The points below describe how we ran the group based on CAS attributes after the initial period of trial and adjustment.

Distributed Control was create by: 

1.    Changing our organization structure from hierarchy to fractals of spirals (groups). We had a spiral with all the main functions needed to reach our goal. Each element on the spiral could be a role or another spiral.

2.    Removing all management positions and replacing them with two main roles. One responsible for task prioritization and the other for resolving internal conflicts and representing the group for broader conflicts. The second role was elected by the group. 

3.    Each role and group had a defined list of accountabilities and assets they managed. They had full autonomy to make their own decisions within their defined assets and accountabilities. One of the main requirements was to notify all relevant roles with their decisions and actions.

We created hybrid teams to demolish silos created over the years. We used Atoms as the basic structure for the hybrid groups. Each group had a nucleus of several roles and people essential to reach the group goal and several electrons that was an integral part of the group, with supportive skill sets. Groups can be a combination of any IT expertise and customers. We didn’t implement this structure just for projects, it was also the structure for other teams within IT.

Part of the accountability of each group was around self-organization. Groups were accountable to define how they work together, how they distribute work, who they were going to add or remove, how they held each other accountable, and any other aspects a classical manager could utilize to organize a group.

Metrics is one of the main feedback loops. Every group had to develop, maintain, and published 5 metrics that would show how they are reaching their purpose. The rule of thumb was two metrics that measure customer satisfaction, two metrics that measure internal improvements and one metrics that measure how the group supported the group it is a part of. The requirement to measure metrics per group created many feedback loops and enabled everyone to see the progress of groups.

Every group had the ability to change roles and operation patterns at any time. We enable them to change continuously and adapt to achieve continuous improvement. Because of external and internal environments continuously changing, we would also expect to see changes with groups. We even measured the ratio of group change to make sure that groups were not falling into equilibria.

We leveraged dependence on initial conditions in two aspects. First, we used it as an operational principle that prepared us for small changes with huge impacts and huge changes with minimal impact. When executing, we expected people to be ready for these two alternatives. The second aspect was related to root cause analysis. We expected people to go back to find when the unexpected change started, what started it, and why it ended up as a nonlinear negative pattern.

Co-evolution was implemented in several ways. The first hybrid groups enforced people to be part of between 5 to 7 groups. It means that the success of one pattern in a group as a result from feedback was shared in other groups and caused replication or mutation of the original implementation. The second method was keeping one metric of each group on the goals of the containing group. This requirement caused collaboration and influence between all subgroups that shared the need to improve one group purpose. The third was defining the conflict resolution between groups at the group that hosted both of them (or their contained groups). This policy creates a lot of transparencies of how groups operate and therefore an influence on other groups.

The state of paradox was more of a guideline for the group who led IT. If we felt that IT was swinging too much towards order, we introduced more chaos into IT. If we felt that IT was swinging too much towards chaos, we injected elements of order into IT. Using feedback, we found out where we were and adjusted ourselves to course correct.

On top of all of those principles, we had to change most of the management and HR processes to accommodate them to support the above principles. Except for base compensation, we changed every process related to management and HR.

What are the main advantages/disadvantages?

Let’s start with the bottom line. We improved every year between 20%-30% on our metrics compared to the previous year. Customer satisfaction, platform SLA/OLA, percentage of finished projects, retention, and budget all showed above average percentages. We reached an 82% success rate for finished projects for the year. The engagement and ownership peaked. People sensed that they can make their own decisions and make mistakes, which saved the company during hurricane Harvey. In meetings and discussions, people said what they thought and didn’t hesitate to say what they believe is not going to work. Obviously, we were far away from being perfect, but as a group we manage to make a significant change. 

Side by side with the achievement, we had challenges. The change was hard for existing senior management, most of them didn’t survive the change. We found that people prefer to resolve issues but not to raise them, especially when the issues were related to others in the group. The change was very hard for personalities that need to be the stars or savers of the day, because the focus was on the group rather than individuals. We had to adjust and simplify the process of resolving conflicts. I personally believe that we failed on this one. 

Our biggest problem was the fact that we operated within a very conservative, hierarchical and command-and-control environment. One of the key elements for the success of such changes is a change into the compensation model that will support this model, we failed to convince our organization to enable and support this change.

Operating IT groups based on CAS principles didn’t resolve all the problems. It significantly improved IT, but it also created a new set of problems we had to deal with. Although this change is harder than the classical agile implementation, it is worth the effort. I encourage you to try adding elements of CAS to your existing agile implementation, measure the results, adjust the principles, and adopt them if you see improvement.