Good Fences Make Good Neighbours

The proverb “good fences make good neighbours” is descriptive of why boundaries are needed in our software designs. Not only as a way to break up the problem and make it easier to comprehend and manage, but also to make the teams working in your company get along better and bring out the best in each other. We unfortunately focus too much on the former at the expense of the latter, where we often break the domain apart haphazardly that not only create a lot of technical coupling, but also setting the teams up against each other, potentially fostering social dysfunction. We need to get better at seeing the whole when designing, realising that when dealing with highly complex sociotechnical systems the old machine model no longer works. Not only is the model wrong; it is harmful.

"Software development is a fundamentally socio-technical enterprise. The strands are not separable in any way.” ―GeePaw Hill

This post is partially taken from a new talk of mine with the same title premièred at NDC London 2021. No recording is available yet, but the slides can be found at SlideShare.

I grew up in the rural west coast of Norway where stone fences are commonplace and I have fond memories of climbing and playing on them, studying animals building their homes there. Also have some less romantic ones of building them, struggling to find the right stones to make a stable and rigid structure, often at places where they are hard to find. All the effort spent paid off when seeing the effect they had, the function they served. Not only are they a good place to dump the stones when cultivating land, but also for marking the boundaries between the farms, what is ours and what is theirs, as well as making sure that the livestock are kept in place. The fences serve a specific purpose wherever they were placed, and a lot of thought and effort was put into them because of their importance and longevity. They were not designed to be walls, mind you, which is just as critical. They should only serve that specific need, while not be too much of a hindrance to other people and wildlife. In Norway and the other Scandinavian countries, we care deeply about what is called “allemannsretten,” which can be translated as “freedom to roam” and it gives the general public right to access certain public and even privately owned land. The fences should therefore not go against this right and some even put in place ladders so people can easily get across.

I think we can take a lot of learning from what these fences are used for when designing boundaries between parts in our software systems, what their specific purposes are and, more importantly, what they are not. As with the mossy stone fences I grew up with, they should be designed and built with care, taking the bigger picture into account, especially the environment and the context in which they are placed. As the American architect and systems theorist Buckminster Fuller said: “You have to remember that any boundary is a useful fiction.” They are not there to be found; they are designed so we have all the power in the world to get this right.

There are several reasons why we find that modularity is useful in our designs, be it to reduce our intrinsic cognitive load by breaking the problem space into manageable pieces, focusing on one bit at the time, or reducing the technical coupling between the parts. As this is inherently a sociotechnical system, there are even more to be gained, especially concerning the need for harmonious collaboration and reduced coordination cost. Well-designed modularity should make good neighbours.

Any system above a certain size will naturally fragment, referred to as self-organisation in complexity theory, and something that you have experienced in your social interaction with people. When a group grows beyond a certain size it breaks up into sub-groups, some even build their own internal culture that sets them apart from the others. The “us and them” perspective is natural and inevitable and can have some dire side-effects if not catered for. A good example of this is political factions that people like the philosopher David Hume and the father of the American constitution, James Madison, where concerned about. When dealing with social systems these natural fractioning forces are strong and we must take them into account when designing for modularity. We must place the boarders so that we not only get an efficient and effective system, but also a harmonious one; where the interaction and collaboration enhance the system, not harms it. As we know, a system is more than the sum of its parts. Make sure then that this is a positive effect, not a destructive one. Also bear in mind Ruth Malan’s corollary to Conway’s Law: “If the architecture of the system and the architecture of the organization are at odds, the architecture of the organization wins.” In a sociotechnical system the socio and technical parts should be jointly optimised.

Any complex system will, by definition, have interdependent parts and therefore have necessary interaction that give the system as a whole its emergent effects and, as mentioned, we want those to be as positive as possible. In order for that to happen we need to get rid of the unnecessary coordination that hinders the systems as a whole. In our line of business that means removing handovers between teams and make them be in control of their own means and outcomes. We want the teams to be autonomous and have agency, avoiding hard dependencies between them. This is why Jeff Bezos of Amazon say “communication is terrible;” not to stop people working together but have as little need for coordination between the teams as possible, e.g. not needing work to leave the team.

By reducing the coordination cost to an absolute minimum, the teams can get better speed by being in control of their own flow and shortening their lead-time from idea to user feedback. The shortening of these cycles will help the team get better stability and agility of both their software and their interaction, adapting better to their context and environment and improving their resilience. This will naturally affect the system as whole as well, where all teams get the same independence and self-governance, making the robustness and scalability of the whole enterprise better. Put in another way: Do not scale agility, de-scale the organisation instead into self-governing parts.

“An organization is a system and the performance of a system depends more on how its parts interact than on how they act when taken separately.” ―Russell Ackoff

Hopefully, it should be self-evident by now that a well-functioning and harmonious sociotechnical system where the whole is greater than the sum of its parts cannot be reached without some holistic design. Random and uncontrolled emergence will not necessarily give that desired result on its own because the old machine model where the system is deterministically defined by its part does not suffice. A systems approach is needed top better cater for highly complex social system with self-organisation, emergence, adaptation, and non-linearity. A combination of top-down and bottom-up is needed, where the former guides the latter in the right direction. I have written about the outside-in perspective before, employing techniques from domain-driven design, business architecture, and user story mapping, but need to take a closer look at how to combine this with the bottom-up design to get the positive emergence happening. That sounds like a good topic to take a closer look at next.