Building a Portfolio Roadmap is a Wicked thing

Why do strategies never seem to work? We have the annual or even quarterly planning ritual; all the money spent on top down analysis using more and more sophisticated processes and models; the vast number of planning sessions, the deference to architecture and looking at business problems through a technical lens, and use more and more elaborate technical solutions (as if that is ever going to not end in tears). Why then do we invariably end up with a portfolio that doesn't deliver the intended value, a portfolio roadmap that those at the coal face just know is wrong? Why do we as supposedly intelligent people continue to get it wrong so often?

To paraphrase Einstein – we can’t solve problems using the same mentality or method we used to create them. If we continue to ignore this big complication, then the increasingly complex models we use to describe our increasingly complex environments will increasingly not work. You don’t solve the problem of increasing complexity with, well, increased complexity.

A bold and arrogant statement? Perhaps – though largely based on the work Stacey (2003), Wheatley (1992), Black (2000) and Morgan (1997). I will explain.

Earlier this decade, IBM did some analysis which resulted in the excellent report "Capitalizing on Complexity: Insights from the Global Chief Executive Officer Study". In it, they surveyed 1500 CEOs and came to the following conclusions; complexity is the biggest problem CEOs face, they are not set up to deal with it, and the problem is going to get worse. In such an environment, and faced with that sort of challenge is it any wonder our annual planning cycle and resulting portfolio roadmap is so often doomed?

Indeed, several CEOs admit that they are confronted with issues that cannot be resolved merely by gathering additional data, defining issues more clearly, or breaking them down into small problems. Their planning techniques don’t generate fresh ideas and implementing the solutions those processes come up with is fraught with political peril.

Why? It’s because when we are dealing with strategic planning and the development of a portfolio roadmap, we are not just dealing with complex problems - they’re “wicked.”

Modern complexity theory provides a lens through which both academics and practitioners can analyse and understand the operation of an organisation down to a granular level, and identify the methods by which change should be delivered. It is concerned with the emergence of order and structure in complex and the apparently chaotic organisational systems.

The unpredictability of change means that organisational leadership cannot effectively manage change, but instead support their organisation on its change journey, releasing individuals to adapt as the organisation moves towards the ‘edge of chaos’ providing the environment for self-management and the avoidance of liminalities.

Complexity theory suggests that the future is unknowable, so the ability to learn is absolutely critical to ongoing organisational effectiveness; navigating the increasingly complex paradox of the desire for stability with that of the need to flex, adapt and change. Too much stability will stagnate the organisation and prevent proactive and adaptive change; too little and the organisation runs the risk of becoming chaotic and impossible to manage.

Complexity theory therefore promotes the idea of organisations as complex adaptive systems which need to respond to both external and internal environments by remaining on the edge of chaos whilst at the same time self-organising and continuously re-inventing the organisational alignment and goals.

However, looking at modern portfolio management theory and practice the term “complex” just doesn’t cut it. We need to go a step further. We need to step into the world of the “wicked”.

Originally introduced in 1967 by C. West Churchman but formally described by Horst Rittel and Melvin Webber in 1973 “wicked problems” are a class of system problems which are ill-formulated, where the information is confusing, where there are many clients and decision makers with conflicting values, and where the ramifications in the whole system are thoroughly confusing. Sound familiar?

Wicked problems are problems with many interdependent factors making them seem impossible to solve. Because the component factors are often incomplete or in flux, and are difficult to define, solving wicked problems requires a deep understanding of the stakeholders involved, and a fresh approach.

In the paper “Dilemmas in a General Theory of Planning,” Rittel described ten characteristics of wicked problems. Taking these one by one and applying to elements of portfolio management illustrates the challenge:

1. There is no definitive formula for a wicked problem – there is no definitive statement of the problem: wicked problems are ill-structured and feature an evolving set of interlocking issues and constraints so every solution that is offered exposes new aspects of the problem, requiring further adjustments, which changes the problem.
2. Wicked problems have no stopping rule - It’s hard, maybe impossible, to measure or claim success with wicked problems because they bleed into one another, unlike the boundaries of traditional problems that can be articulated or defined. Since there’s no way to know your solution is final you can't tell when you're done – there's only a “good enough” solution, which will never be final or optimal. Change constantly stirs this pot of analysis without knowing with certainty when it's cooked. Good portfolio managers know that there is no right or wrong way of prioritising a portfolio. There are only bad, good, or “good enough” ones. Some even end their analysis not by arriving at some logical conclusion but only because they like what they see or they run out of time.
3. Solutions to wicked problems are not true-or-false; they can only be good-or-bad - There is no idealised end state to arrive at, and so approaches to wicked problems should be tractable ways to improve a situation rather than solve it.
4. There is no immediate test of a solution to a wicked problem. You can't fully test the solution. Wicked problems are indeterminate and essentially unique, where endless solutions cannot be fully tested. Portfolio management behaves in a similar fashion. How do portfolio managers know if they did a good job? They can't immediately see the results of their trade-off decisions and what else was affected. A seemingly balanced portfolio that performs well over time may even come as a surprise.
5. Every solution to a wicked problem is a "one-shot operation"; because there is no opportunity to learn by trial-and-error, every attempt counts significantly. There is no trial and error option; every solution is a “one-shot operation” with significant consequences. Some failures are even irreversible. Investment decisions as a by-product of portfolio analysis involve large sums of money. Once invested, it is difficult to retract due to the “sunk cost effects” common in large organisations. Once retracted, additional wicked problems may arise.
6. Wicked problems do not have a set number of potential solutions. - Unlike mathematical puzzles or chess, there are no rules for coming up with solutions. “Anything goes” seems to be the maxim since there are no criteria to exhaust all possibilities using all possible permutations. Portfolio managers or a team of experts rely on good judgment to solve for the “good enough” mix of portfolio components that adequately manages risks without being absolutely sure if they covered all possible options.
7. Every wicked problem is essentially unique - Each problem is essentially unique, and can’t be classified into nice little buckets. There's always something different about a new problem compared to the last one - that's why old solutions won't work all the time. Reusing the logic portfolio managers used in a previous prioritisation exercise can't guarantee success in the succeeding ones. A fresh batch of projects to be prioritised carries its own set of burdens and concerns.
8. Every wicked problem can be considered a symptom of another problem - This is where it gets messy. Focus on solving the higher-level problem means and you just end up with generalised solutions that are often impractical. But if you try to cure a symptom instead you may perpetuate the root cause or worsen it. Hence portfolio managers demanding better software or a more complex model to cure a symptom of failed prioritisation, instead of fixing a broken communication process.
9. There is always more than one explanation for a wicked problem because the explanations vary greatly depending on the individual perspective. For example, opinions on how to prioritise a portfolio based on importance and value vary among portfolio managers because it depends who you ask (and often when you ask them!) - their values and organizational culture differ.
10. Planners/designers have no right to be wrong and must be fully responsible for their actions – is there any wonder why we define and describe ever more complex models to describe business models and crunch the numbers (hoping the numbers are right) in ever more finite details with this in mind?

Doesn’t that all sound painfully familiar?

Conventional doctrines of portfolio management have, as their premise, the reduction and control of disequilibria, unknowns and random events. Strategic management techniques focus on various frameworks where the problems are known and understood, where knowledge and expertise can be applied by using formal mathematical models and conventional tools of linear analysis and where metrics of efficiency and effectiveness can address fluctuations and deviance from expected norms.

However, the issues of knowledge complexity and problem uncertainties raise many challenges for corporate decision-makers. Do managers have the appropriate tools to address unconventional problems? Are they tied to conventional thinking, assumptions and approaches to strategic decision-making? In their discourse, Lafay and Martin (2013) describe how to approach this dilemma:

• be clear about what business to operate in;
• where to operate;
• how to win; and
• what competencies are needed to consistently outperform the competition.

This approach forms the basis of the core theories and models taught in business schools. Corporate strategy models now are more comprehensive, shifting the emphasis to include not just corporate positioning against competitors, but also encompassing people issues, network alignments and the management of ideas and innovation in uncontested markets. In effect, trying to make sense of the increasing complexity of our businesses, our people and the environments we operate in.

Fundamental questions remain however: for example do these models provide the right strategic approaches needed for all markets, strategic problems and firms?

There are massive inconsistencies between model assumptions – openness, competition, transparency, executive responsibility and low risk – and the reality of decision premises, a form of hypocrisy between mission goals and actual behaviour. Most theories of strategic decision-making apply premises and assumptions of rational behaviour and stable systems. In finance and the world of banking, actual strategic behaviour differs wildly from the rational model of perfect competition, where all the variables are known, where information is free and available and where computation of strategic choices and their consequences are examined in minute detail. Indeed, the real world of corporate finance is the exact opposite. More generally, strategic approaches to complex problems are misaligned with the problems themselves and potentially make firms ill-equipped to address wicked problems with simplistic approaches.

So what can we do about it? How do we change? How do we navigate through the challenges of wicked problems and increase the chances of arriving at a portfolio roadmap that’s based on a strategy that better reflects reality?

Let’s start from these statements (taken from Carrie Foster’s Five Core Theories in Organisation Development) – all of which, to put it mildly, are thought provoking:

1. Change can’t be managed in a complex or wicked system
2. Change must be supported
3. Leaders must encourage people to learn how to adapt and flex
4. Open and honest connection between the different parts of the organisation is essential for self-organisation and embracing diversity of thinking, ideas and approaches
5. Feedback loops and Information flow is essential to prevent the organisation from falling into chaos.

So how exactly should we approach wicked problems such as those encountered in portfolio management – specifically around strategic management and developing a portfolio roadmap that delivers real value?

To get slightly academic and think theoretical for a second (I'm doing a PhD - work with me here!), systems thinking is the process of understanding how components of a system influence each other as well as influence other systems. That approach ought to help surely? On the other hand, an agile approach is an iterative way of solving a problem, giving a collaborative environment that breeds the ability to be efficient and effectively meet the stakeholders’ changing requirements. That ought to help too, surely? So why not combine the two and have a systems approach with an agile bent?

Here I can lean on others who have had thoughts on a similar approach. We can break this down to five lines of attack: 

• Jay Wright Forrester, a pioneer in computer engineering and systems science suggests breaking down information into nodes (chunks of information, for example objects, people or concepts) and links (the connections and relationships between the nodes). Doing so utilises systems thinking when faced with a wicked problem makes our private mental models (our representations of the external reality) visible to the outside world
• Take that a step further and visualise the information. Sketching and placing information into a physical space helps both you and your team take in and understand the systems at hand as well as the relationships within and between them.
• Wrap that around being collaborative and including stakeholders in the process. Sharing your mental model helps other people build on your ideas and vice versa. Creating physical drawings and grouping notes to produce different models allows the team to describe and detail several points of view.
• Importantly, release solutions/ideas/proposals quickly to gather continuous feedback. Feedback of success will help with solving problems we don’t have one right answer for. The more feedback you gather from your users and stakeholders, the more guidance there is to get to the next step and the more appropriate the strategy and roadmap will be.
• And finally – iterate. With each iteration, you have the chance to utilise feedback and determine what changes are needed to further improve the solution for your wicked problem – in our case, developing a portfolio roadmap based on a meaningful strategy.

So with that in mind, if we circle back and look at Rittel’s ten characteristics of a wicked problem, it looks like we should be in business. At the very least, we will have gained recognition that traditional approaches to strategic planning and portfolio management, and therefore building a portfolio roadmap that delivers the value we expect, ain’t going to cut it. As a famous late 20th century philosopher very aptly put it – some people feel the rain, others just get wet.

The philosopher’s name? Bob Marley.