Innovation and Insurgency

Why is it so hard to do innovation in a big organisation and what can we do to ensure that small organisations are able to keep innovating as they grow? We can learn a lot from the historical interaction between hierarchies and networks

Throughout history there’s been a continuous interaction between established, stable structure and small, faster moving, destabilizing innovation. This happens at every timescale from the slow march of evolution over millennia, to the exponential pace of our current economic changes.

It is not the strongest of the species that survives, not the most intelligent that survives. It is the one that is the most adaptable to change.” - Leon C. Megginson (NOT Charles Darwin)

This quote is often miss-attributed to Darwin, but it actually came from a marketing professor in the 1960’s [1]. Leon C. Megginson actually said that “those who survive are the ones who most accurately perceive their environment and successfully adapt to it.”

Sounds like someone else I know of…

To comprehend and cope with our environment we develop mental patterns or concepts of meaning. The purpose of this paper is to sketch out how we destroy and create these patterns to permit us to both shape and be shaped by a changing environment. - Col. John Boyd

More on this later!

Commandos - insurgents since WWII

One example of small, agile groups being deployed to disrupt larger structure is the formation of the Commandos, SOE and other special operations units in the Second World War. This is a text book case of an innovative insurgency on three levels:

• The resistance of the establishment it grew within.
• Its use in asymmetrical attacks against the weak points of a much larger foe.
• The reliance on small, highly trained, SEPARATE teams of people

The SOE - Churchill’s Ministry of Ungentlemanly Conduct

The story of the rise of unconventional warfare despite massive resistance from the War Office is told in the fantastic book Churchill’s Ministry of Ungentlemanly Conduct. It’s extremely well written and entertaining, but the main takeaways for me were:

• The need to build SOE outside of the slow moving hierarchy of the War Office
• R&D without a specific goal (at least initially).
• Deployment of capabilities against the weak points of the enemy network.
• Team built from diverse backgrounds. e.g. Chief engineer was a caravan builder.

Commandos, Infantry and Police

Robert X. Cringely noticed a pattern in the evolution of tech start up in his 1996 book Accidental Empires. He identifies 3 phases of development Commandos, Infantry and Police.

COMMANDOS take the beach. They figure out how to smash the status quo and find opportunities to exploit. They have and need very little structure, iterate quickly and make things happen.

INFANTRY are the second wave. After the commandos have gained a beachhead it’s down to the infantry to build upon what they’ve eked out. The commandos will likely leave a big mess which will need to be sorted out and systemised. So there are more infantry and they need correspondingly more structure and process.

POLICE Come in when there’s a need for even more stability and regularity.

Simon Wardley built on this analogy when we came up with 3 different types of team within a business and placed them on top of his axis of evolutionary development. He observes that as products, services and components move from genesis to commodity the types of teams you need and the way you work changes.

Commandos are pioneers, Infantry are settlers, Police are the town planners Simon Wardley’s “Poineers, Settlers and Town Planners”

Social Inflection points

In my last article about Social Inflection Points I had a table of group sizes. I want to recreate this here and add Commandos Infantry and Police to the mix as a new column (CIP).

The CIP column is the size of organisation where these stages of team evolution will start to appear. So below a tech company below ~30 people is made up mostly of commandos or pioneers. Infantry or settlers start to be added in the journey to 100 or so. Police or town planners come into play anywhere from 100+.

What we’re also likely to see as an organisation moves along the continuum from early stage commando unit to established company is the formation of a hierarchy. Even in “leaderless” structures like holacracy, there’s still a formal structure that emerges. This has to happen in one form or another because as a group grows past 15 or so people it becomes untenable to scale the 1-1 communication between members.

Why?

Metcalfe’s Law

The of number of connections in a network is proportional to the square of the number of nodes.

Metcalfe’s law from Sketchsplanations

So as networks expand, spontaneous order starts to emerge.

What’s unfortunate for growing tech companies, is that his need to create structure and order starts to emerge just when they want to really start to scale. So what can companies do about this? Can we achieve economy of scale, while still achieving agility?

Organisations in today’s world are subject to increasingly punctuated equilibrium [2]. It’s simply a fact of life.

The best companies in the world either consciously or unconsciously recognize this and position themselves in such a way that they become anti-fragile to dislocations.

They do this by constantly, often uncomfortably, innovating. Disrupting themselves so that an external disruption is part of everyday life. A bit like the netflix chaos monkey, but for everything! This is exactly what google did with the establishment of alphabet.

Further reading

There have been a wealth of books that have dug into these phenomena. Way too many to provide a full treatment of here. I’ll include a list of some favorites at the end, but wanted to say a paragraph or three books in particular. The Commando Development “must read list” if you will :-)

Team of Teams - Gen. Stan McChrystal [3]

Team of Teams is the story of the fight against al-Qaeda in Iraq. This was really the first time that we saw a loosely organised, ideologically aligned, highly networked modern insurgency.

The book (and its follow up One Mission) contains a fascinating insight into the changes a fairly rigid, hierarchical organisation had to make in order to be able to counter an enemy that was innovating daily.

From hierarchy to Team of Teams Beau Gordon on Medium

Main takeaways:

• The use of cross functional teams to keep the size of sub networks to a minimum.
• Shared consciousness to keep everyone aligned.
• Empowered execution to ensure decisions are made by those "in the fight".
• The need to be able to change at least as fast as the environment you operate in.
• The value of hierarchy for economies of scale. e.g. payroll and admin.

Loonshots - Safi Bachal [4]

Just like how changing the temperature of a fridge by one degree can change water from a solid to a liquid, “Loonshots” explains how making small shifts in company structure can lead to dramatic outcomes that can alter company’s future. The book distills these insights into lessons for creatives, entrepreneurs, and visionaries everywhere.

Main takeaways:

• Separate the Commandos and the rest (artists and soldiers in the book).
• Create dynamic equilibrium - love them equally.
• Tailor the tools to the phase.
• Appoint, and train, project champions to bridge the divide (boundary spanners).

The Square and the Tower - Niall Ferguson [5]

Niall Ferguson takes a look at the last 500 or so years of history as a battle between networks and hierarchies. The square (market squares, town squares) represent the messy, organic networks humans form. The tower represents hierarchies that rise and fall as they exert control and get over thrown. This rise and fall is precisely what I mean when I say innovation and insurgency:

• An innovation develops allowing development of hierarchy.
• The hierarchy exerts control, driving the system toward regularity.
• Insurgents push the boundaries of control, innovating their own advantages.
• And so it goes.

Some of my favorite excerpts:

Networks are not static, but dynamic. Whether they are random or scale-free, they are prone to phase transitions. They can evolve into complex adaptive systems with emergent properties. Very small changes -the addition of just a few edges – can radically alter the network’s behavior.

By the same token, the seemingly rigid structures of a hierarchical order can disintegrate with astounding rapidity.

The problem is that networks are not easily directed ‘towards a common objective … that requires concentration of resources in space and time within large organizations, like armies, bureaucracies, large factories, vertically organized corporations’. Networks may be spontaneously creative but they are not strategic.

It distinguishes the long epochs in which hierarchical structures dominated human life from the rarer but more dynamic eras when networks had the advantage, thanks in part to changes in technology. To put it simply: when hierarchy is the order of the day, you are only as powerful as your rung on the organizational ladder of a state, corporation or similar vertically ordered institution. When networks gain an advantage, you can be as powerful as your position in one or more horizontally structured social groups. As we shall see, this dichotomy between hierarchy and network is an over-simplification.” Is it better today to be in a network, which gives you influence, than in a hierarchy, which gives you power?

The crucial incentive that favored hierarchical order was that it made the exercise of power more efficient: centralized control in the hands of the ‘big man’ eliminated or at least reduced time-consuming arguments about what to do, which might at any time escalate into internecine conflict.”

Other books and resources

There are a million more books and other resources I’d like to dig into, but these will have to wait for another time. A by no means exhaustive list:

One Tribe at a Time - Maj. Jim Gant on US Special Forces’ experience in Afghanistan

Network Science - Albert-László Barabási

Churchill’s Ministry of Ungentlemanly Conduct (mentioned earlier)

Reinventing Organizations: A Guide to Creating Organizations Inspired by the Next Stage of Human Consciousness - Frederic Laloux, Ken Wilber

Holacracy: The New Management System for a Rapidly Changing World - Brian J. Robertson

Skunk Works: A Personal Memoir of My Years at Lockheed - Ben R. Rich

Conclusions

We exist in a complex web of organically structured organisations, ecosystems and the links between them.

The connectivity boost ushered in by the rise of the internet, and in particular social networks has made it more important than ever to understand how these systems function, interact and break.

We can look at insurgencies as a proxy for innovation in organisations that have become overly linear and structured.

If your organisation has become ossified and can no longer react to the disruptions and dislocation of the market, changing that had better be in your top 5 priorities.

1. https://www.darwinproject.ac.uk/people/about-darwin/six-things-darwin-never-said/evolution-misquotation
2. Punctuated Equilibrium posits that most social systems exist in an extended period of stasis, which may be punctuated by sudden shifts leading to radical changes. In our VUCA world these shift come more often and are of greater magnitude.
3. https://www.goodreads.com/book/show/22529127-team-of-teams
4. https://www.goodreads.com/book/show/39863447-loonshots
5. https://www.goodreads.com/book/show/35629744-the-square-and-the-tower