See the Forest for the Trees

When developing your software products, be it coding, testing, user experience, product management or all the other elements required to solve a customer need, do you have a good understanding of what the rest of the people does to make that happen? What about the other people in your organisation, maybe working on different products or even other legs of the customer journey like sales, customer service, billing, and operations? Do you see how you fit in to the big picture, what your contribution is to the company vision and strategy? I suspect most of us neither have the time nor the opportunity to get a wider view, focusing on our little part of the bigger system instead and making the best out of that. The problem is that it gives us no assurance that what we do actually provides value – it may even work contrary to others, creating an overall negative result. We know that a system is supposed to be more than the sum of its parts, but how can make sure that the sum is positive? That is hard when we cannot see the forest for the trees.

“A system is more than the sum of its parts; it is an indivisible whole. It loses its essential properties when it is taken apart.” ―Russell Ackoff

Even though Aristotle back in antiquity hinted at the fact that the whole may be greater than the sum of its part, our history has been mostly about understanding anything around us by breaking it apart and studying its elements in isolation. This has been the golden standard at least since the scientific revolution and so ingrained in us that we all think analysis is the only way to insight. The success of this approach is unquestionable, with all the innovations and progress we have made by employing this reductionistic thinking, but it rests on the simple notion that the parts explain the whole. In many situations this way of thinking is sufficient, that the explaining power of this model is good enough to carry us forward, but the usefulness of this model has limits as any model has. At some point all the simplifying assumptions made, be it removing the environment by closing the system under study, assuming homogeneity when there is huge variety, or focusing on some of the parts only, will hinder further progress and deeper insights. Complexity is all around us, especially in any system involving humans such as software development, and we need better tools for handling those. Reductionism and analysis alone will simply not do when dealing with so-called wicked problems.

Crisis frequently has a silver lining in that it brings about leaps of innovations, be it global pandemics or the world wars. The latter brought about systems thinking, where one of the manifestations of reductionistic thinking, the hard division between the different scientific disciplines, where challenged and people started collaborating across the different profession in order to better deal with the rut humankind was in. Commonalities was discovered and new ideas was forming that proved to be better tools for the complexity of the modern world. Systems was the new perspective and understanding that it consists of a lot more than its parts; there was also interconnection between the parts and the systems had functions and purposes of their own. We learned that the old machine model of the world needed replacing with a more holistic systems model.

Software development as a profession has strong lineage back to the industrial era and its engineering way of thinking. We still frequently work in hierarchical bureaucratic organisations; we employ techniques from the production lines to optimize flow and efficiency; and we treat all input as expendable resources – even humans. This is a brittle and even harmful reductionistic model that does not deal well with the complexities of modern enterprises, where there are so many essential parts with often diametral purposes and so many interconnections, both inside and outside, that we are barely coping. The systems age brings new ideas to the table, where this complexity is embraced, not supressed.

One core idea to systems thinking is the addition of synthesis to our toolbox. Not as a replacement of analysis – that is still a great tool – but more as an extra dimension to explore that helps dealing with complexities. In a way it goes in the opposite direction to analysis: instead of decomposing a problem, then explaining the parts and using that to explain the whole, synthesis starts with identifying and explaining the containing whole of the problem at hand, seeing how it fits in with that environment and explaining it in that context. In a sense you go wide before going deep; you synthesise first, then analyse second.

“Analysis focuses on structure; it reveals how things work. Synthesis focuses on function; it reveals why things operate as they do. Therefore, analysis yields knowledge; synthesis yields understanding." ―Russell Ackoff

In software and IT we have gotten a good handle on analysis, breaking apart a problem domain into neat parts and treat them separately. We believe that is the way to handle complexity, “divide and conquer.” What we need to do first though, at any level of that decomposition, is to first understand the containing whole and see the role, purpose, and interconnection all the parts have at every level. The thing is that the parts not only explain the whole, the whole also explains the parts. When building a new software product, you need to understand what context it is part of, what the environment is, be it customer segment, user profiles, business context, its goals and purpose. You need to get the outside-in perspective first – only then are you ready to truly get a grasp of the complexity your product has to deal with and designing systems that are both fit for purpose, resilient, and sustainable.

"Every software-intensive system has an architecture: some are intentional; a few are accidental; most are emergent.” ―Grady Booch

This may sound like we have to go back to big-design-upfront and long lead time of the waterfall era, but as systems thinking showed, synthesis and analysis should be combined and a good way of doing this is iterating between the two. Preferably frequently, shortening the feedback loops of learning how the product performs. The following illustration taken from a paper by Barton and Haslett illustrates this well.

This way we can combine the holistic top-down approach with the agile bottom-up principle of emergent designs, where we together iterate between synthesis and analysis as frequently as we can. Eduardo da Silva has a great post and presentation on how this can be done at scale: Evolving Tech-driven Orgs Using Sociotechnical Architecture.

We need to renew our own celestial model, so to speak, realise that the mechanistic view of the world no longer suffice, that the complexity of the social system our software products are inherently a part of no longer can be dealt with deterministically. This will probably require a monumental shift for many, not only in how we deal with problems and develop software solutions, but also how we organise ourselves, how we cooperate, and how we collectively share a common purpose. Only then can we hope to be more than the sum of the parts; let’s be the product of our interactions.