Where is your IT Department's Supercarrier Complexity

So far, this month's top Tweet on my feed is one that calls out the failures of the US Navy's newest "supercarrier" -- the USS Ford -- for its inability to safely and reliably support jet take-offs and landings. The pull quote from the tweet is:

"In short, the next-generation aircraft takeoff and landing systems 'remain unreliable and break down too often...'"

That's an important task! To channel a rather well-worn meme: "You had ONE JOB!"

But anyone working on large projects of any kind (including software projects) knows how common this problem can be. The ability to start a large project with good intention far outstrips our ability to complete a project on-time, on-budget, that actually does what was designed to do in a safe and reliable way. 

So, if this problem is so recognizable, why does it keep coming up so frequently? Is there something inevitable about creating large scale projects? Is it possible that doing something "big" is actually not possible? 

Maybe there are some "laws" we keep breaking when we try to create something large. And maybe someone has already told us this?

Conway's Law

Mel Conway is credited with authoring what Fred Brooks calls "Conway's Law". Although first penned in 1968, Conway's Law has gained quite a bit of currency in the last few years as people rally around the notion of DevOps and Microservices in the software industry. You can follow the link to read the full text of his "law" but essentially, Conway tells people that your output is governed by the way you communicate as a group. If your communications are chaotic you'll get chaos as a result. 

Some people have tried to "operationalize" Conway's Law into things like "doing a reverse Conway" or using his observation as proof of a particular management or organizational style. I don't favor that type of approach but i do find that, as Conway stated more than one half a century ago, your output is governed by your organizational dynamics. If your company puts lots of effort into "managing" the group, you'll get the expected results -- a "managed software output."

Christensen's Dilemma

Here's the way Conway reacted to my tweet on the USS Ford:

Growing complexity painting itself into a corner. Can't help thinking of Clayton Christensen's "The Innovator's Dilemma". -- @conways_law

I found it instructive that he called out Clayton Christensen here. Christensen is another name that has been gaining popularity in the software world. But Christensen's thinking and writing have been embraced more by the business side of IT whereas Conway has been championed quite a bit by the technical side of IT. 

What intrigues me about both Conway and Christensen is that they both have a history of devoting time and effort to subjects "outside" the IT space. Conway's original writing on the subject ("How Do Committees Invent?") was meant to apply to much more than software and his current musings focus on the workings (and failings) of government and society. At the same time, Christensen's bibliography includes books on the state of higher education, world poverty, and his own personal strategy for life in general. 

Both these people find inspiration, evidence, and challenge for their way of thinking in all aspects of their lives. And whether you are creating a mobile app for your own personal use or launching a supercarrier for one of the world's largest navies,  thinking in terms of how systems interact with each other is a key to success.

Everything is a system, and there are systems within systems. It's turtles all the way down!

Gall's Law

John Gall was another person who thought in systematic terms. And, like Conway and Christensen, Gall has been quoted often by IT professionals. But unlike Conway's math degrees and Christensen's Harvard Business School pedigree, Gall comes from the world of biology. He was a practicing pediatrician and highly-regarded in his field. 

And, like Conway and Christensen, Gall saw a world of interacting systems all around him. His 1977 book "SystemAntics" lays out a rich and engaging point of view where one of the key features of all systems is failure itself.  In his world, our inability to accept the fact that healthy systems are constantly experiencing failure at some level is the primary reason we are terrible at designing, implementing, and maintaining systems of any level of complexity.

Gall's Law (as it is informally known) states it bluntly:

“A complex system that works is invariably found to have evolved from a simple system that worked. The inverse proposition also appears to be true: A complex system designed from scratch never works and cannot be made to work. You have to start over, beginning with a working simple system. -- John Gall, Systemantics

Kelly Johnson's KISS

And it is here that we finally land on what Galls sees as a fundamental truth that all of us must face when we are attempting to create systems. A truth summed up in another well-worm meme: "Keep it simple, stupid".

The KISS principle originated with legendary system engineer, Kelly Johnson while working for US defense contractor Lockheed. Johnson was involved in the design of the Blackbird spy plane and several other groundbreaking military aircraft of the 1960s and 1970s. 

Like Gall, Christensen, and Conway, Johnson's view of the world was shaped by simple strategies and applied to more than his job title at work. Known affectionately as "Lord of the Skunkworks", Johnson's "14 Rules of Management" are a treasure trove of leadership and tactical thinking that can be applied to just about any aspect of a person's life.

Like Gall, Johnson knew that keeping things as simple as possible was essential when tackling a big problem.  And, like Conway and Christensen, Johnson recognized that the way you organized the work was just as important as the work you were attempting to do.  

What have we forgotten to learn?

So, if I can cite just a handful of individuals from the last 60 years of engineering history who understood both systems and the people who work with them -- individuals who offer successful approaches to solving big problems with small solutions -- why are we still hearing about large-scale failures?

I can only attribute this to our ability to forget the valuable things we've already learned in the past. Or, more directly, our failure to pass along the hard-earned wisdom of those who came before us. But then, that's not a surprise, iIs it? I mean failure is a feature of every complex system!

For more, subscribe to my youtube channel and substack newsletter.