Architecture, Strategy, Systems Thinking, Agile

This article contains my reflection on another blog article by Svyatoslav Kotusev , forwarded by Daniel Lambert, M.Sc. – knowing Daniel’s posts, it’d be interesting to know his opinion on the forwarded blog, too. Nevertheless, the concerned article is a good piece of writing. As a non-native speaker I enjoy its flowery language. Perhaps more importantly, I can subscribe to practically everything described in the analytical part (the first half or so of the article). The last third, however, raised two questions in me. The first one was: why? And the second: where is the flaw? I couldn’t help it, but it associated in me siloes and Conway’s law . I think I have found an answer at least to the second question. Well, if not a flaw, then certainly a challenge.

I confess that I belong to those believers of the “chimeric idea” that architecture – not necessarily outputs of someone with the “architect” title – should be based on a strategy. The only explanation I have is based on other torn down terms – systems thinking and agile.

The flaw/challenge can be articulated easily. It occurs in two spots in the text:

• Strategies of business units are normally developed by senior executives, almost always without any involvement of enterprise architects, and ensure the linkage of business unit plans to toplevel strategy.
• This information (overlapping needs, ... common problems and ... redundant systems) is then incorporated into the resulting roadmaps via proposing initiatives beneficial for the organisation as a whole.

Considering that a (top-level) strategy is “vague” or even “missing altogether” it could be quite challenging to link it reasonably to anything. The unspoken truth is that decisions in such organizations are probably made per the HiPPO principle, per collective voting (no one’s to blame), or similar. It does not necessarily mean that they are bad, they just – neglect something that might possibly make them a bit “better” (I will try to explain what I mean by that laterin the text). And yes, one sign of such organizations usually is that they provide only limited autonomy, or overall are not capable/interested in hearing from bottom-up, i.e., about new opportunities and threats, esp. those related to decisions made. They also usually provide quite a detailed description of what, but almost no guidelines of why.

However, the need for a strategy didn’t disappear – it was only factorised or even displaced , in its full Freudian meaning, to a smaller group of decision makers. It’s fair to admit that even having no strategy is actually a strategy (sometimes referred to as flow with the tide or organic growth, usually requiring a minimum of changes to the current state; hence its popularity). If that works, even when it from-time-to-time requires releasing few baits that seem to claim otherwise, why not? The problems may start when the tide flows the organization in a direction where it does not want to be. Then, it can be already too late, because such an organization might be missing the necessary skills to spot warnings and perform a change. In one organization they expressed that top management tried to turn the steering wheel, but the organization did not change its previous course whatsoever. As you like it.

Architects do not create strategies. Yet, they can be among their first consumers and if an organization is willing to listen to bottom-up, they can provide early and fast feedback – test the strategy, esp. for practicability. I have been to many organizations, either as an employee or as a vendor. Even in the most spastic ones, architects were often quite smart and willing to contribute. And unlike external consulting companies who are usually called for if problems occur, they know the organization and are already on the payroll. If for no other reason this could be an argument for considering whether they could not be engaged more in strategy creation or updates. When they play this game together with business, they will quite naturally use the learnt facts even in their “architectural” work, if such a separation is still needed.

Architecture

Personally, my first encounter with architecture was some 15 years ago, in the 8th year of my profession IT career. At that time I had two quite dissimilar sources of information, a book Software Project Management: A Unified Framework by Walker Royce (you may also check the IBM’s RUP ) and Martin Fowler’s article Who Needs an Architect? , which also discusses what architecture or a role of an architect is, concluding that Architecture is about the important stuff. Whatever that is.

Many years later, when I had had a chance to meet and work with a dozen of architecture departments in various organizations, I could confirm that a lot of them resemble Fowler’s Architectus Reloadus archetype. They are primarily rule driven. The rules are not a problem. The problem is the lack of autonomy not allowing to rely on architect’s or someone else’s common sense and experience to evaluate which of them are meaningful in a specific situation. So, to be on the safe side, all of them must always be applied. And as the time goes, the list of rules tends to prolong as no one has criteria to classify some rules as obsolete and abolish or modify them. One example can be a rule that an architecture committee, a process that took at least 50 man-days of highly paid people, must approve a solution architecture containing any new system. This rule was created at the ERP times, when practically any new system represented an effort of several man-months at minimum, and its purpose was mainly to identify functional redundancies or non-functional risks that a suggestion could contain. However, the system under scrutiny was a SaaS service for car fleet management with operational expenses such as 50K EUR per annum. Yet, it had to undergo the heavy-footed approval process that blew up its profitability for years ahead. Give to Caesar what is Caesar’s.

I have one experience which is a kind of exact opposite – they might be considered as one who had asked the Fowler’s question and answered No one. This organization, under so-called agile transformation, dissolved its architecture department, gave the full autonomy to “tribes”, led by former Board-2 directors (in a very bureaucratic organization), and hoped for 20 per cent savings in development costs, because they were agile, weren’t they?

They did have a strategy, or rather the main strategic objective, focused on (measurable!) improvements of the NPS . As you can probably imagine yourself, the results were almost an absolute disaster. As one HR rep expressed it, “we smoke the dope, but don’t deliver.” That wasn’t entirely true – they did deliver, but only when the scope of delivery didn’t exceed the realm of a single tribe, or when (very rarely) two or more tribe leads could find an agreement. One caveat was in limits caused by budget distribution that still followed the old rule, tell me (in details and up-front) what you’re going to do, and I may give you money for that. The other was in protecting their own realms. In another organization they tried to improve their profitability that suffered from high commissions paid to reselling partners who generated more than 70 per cent of revenue. The head of distribution, whose position was strong due to the results, turned down every single proposal for improvement – because his compensation was based on turnover.

Architects are just a nuisance, clever enough to grasp these facts, and sometimes even with contacts that might force managers to RESPOND, so better keep them away.

Well, if you think that this could actually be useful, you might want to take a look at architecture artefacts, such as motivation views , or business capability or value streams heat maps before you make a decision, because those visualization techniques might be instrumental in structuring both known facts and assumptions, estimate complexity as well options for solution, and unveil potential conflicts, incl. stakeholders who might be able to resolve them. By doing so, they may reduce the uncertainty, hence make the decisions better.

It’s true that neither Royce, nor Fowler wrote about this type of architects. Eben Hewitt did. In his book, Technology Strategy Patterns , he describes his personal view on architecture and architects. He starts with an observation that architecture begins when someone has a nontrivial problem to be solved. The product management states what must be done to solve the problem, and the architect describes how to realize that vision in a system. He continues with a definition of architect’s work as the set of strategic and technical models that create a context for position (capabilities), velocity (directedness, ability to adjust), and potential (relations) to harmonize strategic business and technology goals. More precisely, an architect is concerned mainly with entropy (uncertainty), specifying the non-functional requirements, and determining trade-offs (trading a problem for one that you would rather have). At least two out of these three concerns cannot be done without a holistic approach, or at best only in a very limited way. And he concludes that absent a strategic mindset, many technologists left to their own devices create what amounts to little more than shopping lists of shiny objects. We hear this frequently described as “a solution looking for a problem”. Again, something I witnessed in quite a few organizations, at the business unit level.

Strategy, Systems Thinking, Agile

The caveat with strategy-based approaches lies in the strategy. I agree that not every document headed with a word “strategy” in its title is suitable as a basis for strategy-based approaches, not to forget missing strategies. However, let’s turn this question the other way around – what documents are?

In this chapter I will rely heavily on another book, Patterns of Strategy by Patrick Hoverstadt and Lucy Loh. This book starts with rumination on why so many strategies fail, quoting claims by Henry Mintzberg (“around 90% fail”) or Russell Ackoff (“98% not implemented”), providing a possible answer by Gary Hamel: The dirty little secret of the strategy industry is that it doesn’t have any theory of strategy creation. Then, it analyses traditional approaches towards strategy creation, concluding with a finding: It assumes that the destination of our strategy exists – as if it was an actual place like the seaside. But it isn’t. Strategy is about the future, and the future doesn’t exist; it gets created, and it gets created by the various players in our strategic environment – including us. This isn’t just a semantic or academic point.

The proposed way out is a different perspective on what a strategy is, explained in the context of systems thinking, esp. autopoiesis or structural couplings, terms coined by microbiologists Humberto Maturana and Francisco Varela. Even other thinkers, e.g., Whynde Kuehn , noticed that nature provides many useful examples and inspirations, such as closed loop systems, diversity and redundancy, learn and adapt, or principles of self-management. Autopoiesis (self re-creation) is a process how to maintain or develop self through (structured) couplings with other players as well with the surrounding environment. These couplings are assessed from three dimensions (fit, power and time) that can be further divided into five elements and have altogether five enablers.

A strategy is, then, defined as changing our fit with the environment to our advantage by differential use of power and time.

In a nutshell, it means that an organization will change one or more of these parameters in one or more of its couplings, incl. creation of new couplings or termination of existing ones.

The advantage is viewed mainly from the perspective of higher value (or lower value for competitors) but can also mean other advantages such as lower risk or more options for maneuvering in the new strategic position. Value is another elusive term that can be linked to high level motivation drivers, such as the organization’s mission, found in value streams, or analyzed at the very atomic level of an individual exchange between two coupled stakeholders. Either way, there should always be one or more metrics that serve as a sort of proxy proving that a value exchange is flowing, or even quantifying how much.

In my opinion, already this is a great contribution, because it shows a direction how a strategy that doesn’t fail in most cases could look like. Of course, it might not convince someone who considers systems thinking as unable “to provide any actionable suggestions”, have “nothing to show in terms of valuable recommendations for EA practitioners”, or “borders with pseudo-science”. The authors are fully aware of risks of failure, formulated as: one is concerned with the probability that the situation we are fitting to will change to deny us the benefits we were banking on, and the second is that the situation remains stable and the benefits are there, but someone else moves in and deprives us of the value and position we need.

The authors also provide around 80 patterns, such as the mentioned organic growth, they could observe and verify in the real life and that are repeatable, sometimes elaborated to the level of capabilities needed for executing them, the maneuvers, incl. indicators of success for each stage, and turning points. They categorized them among categories: competitive, collaborative, strategies for small organizations, defensive, growth, market-changing, supplier, herd management, and “cunning plans” (tricks and deceits).

A flaw in the second article is that it positions (systems) thinking and communication as if they were in contrast. It is like asking a question who is more important – an ambulance driver or a surgeon. Yes, both are needed, and, in my opinion, both belong to the core competences of an architect. In fact, there are four such competences, hidden under the abbreviation OODA – Observe – Orient – Decide – Act. Note that its author, John Boyd , considered: “The second O, orientation—as the repository of our genetic heritage, cultural tradition, and previous experiences—is the most important part of the O-O-D-A loop since it shapes the way we observe, the way we decide, the way we act.”

The OODA loop is another theoretical concept utilized in Patterns of Strategy. Its role is to proceed from a strategy to a plan (something that is often mistaken for a strategy). The process looks as follows:

1. Assess your current strategic position – its strengths and weaknesses by modelling each structured coupling in turn, using the 3 dimensions.
2. Define your strategy – identify your options (what parameters in which couplings can/should be changed) and choose the preferred one.
3. Create a plan - The plan confirms the set of manoeuvresto be executed and the changes in capability required to deliver them. These changes in capability feed directly into the plan for strategy execution. You can then build a measurement system to monitor the progress of the strategy and its effects on both you and on the other actors involved in structural couplings with you. Once the plan is complete, you also use a simple approach to create a clear and powerful communication message for the strategy.
4. Test the strategy – for the robustness of your thinking, the coherence in your position across the different couplings, and practicability: can you actually do it and is it likely to work?

This loop is by nature iterative – and it expects failures and can react on them. After each executed set of maneuvers, it can evaluate the results with pre-planned measurements and targets, but also consider new learnt facts and incorporate them into either the plan or the whole strategy: Whenever each player makes a change that affects the strategic position, it is important to review and refresh our strategy and renew our execution, based on what we now know about the environment and the disposition and actions of other actors in it.

This is where agility comes in useful. Agility is one of strategy enablers and is defined in a slightly narrowed meaning as an ability to reallocate resources. Other enablers are Critical Mass, Cycle Time, Foresight, and Change Rate. The Agile Manifesto works with at least other two.

Holistic architectural artefacts can be used across all those four stages – value streams can help to find the right structured couplings, motivation views can identify options as well as possible conflicts, heat maps may suggest the preferred option, business capabilities and value streams can determine the scope of maneuvers in a plan or be used for its verification.

Can this process be improved? For sure – for instance, in its Orient stage it may consider a quantified risk of the identified options and consider whether for a decision to be made it is not worth to reduce the uncertainty first. This is what the Applied Information Economics by Doug Hubbard may help with.

However, there is one battle that this approach must absolutely win to succeed. I will use one more quote from Patterns of Strategy describing it as a closing clause: Traditionally, strategy is developed on an annual cycle or even longer but, of course, the reality of strategic actions by us and others don’t fit so conveniently into an annual timetable. The most difficult aspect of this to change is the annual budgeting cycle, the timing of which constrains the rate at which resource can be redeployed to effect a strategy – literally, a rate-limiting factor.