Why even the best Innovation Systems may fail

Summary

Good ideas aren’t evenly distributed. And although anyone can turn out to be a genius, not everyone has what it takes to be one. Only a few innovation systems and organizations in the world understand and manage this well.

New insight

Lately I have had the privilege to study for professors Jesper S?rensen and Willam Barnett as part of the Stanford LEAD program. One of the takeaways for me has been a more nuanced understanding of why failure is seen as something good in organizations. If you know me, or have read my previous rant against the cult of failure, or perhaps have seen my old talk on the subject, you know that I think something is missing in the failure-leads-to-innovation-equation.

Although the course was packed with good stuff, one particular framing of the exploration challenge for organizations looking to innovated was particularly interesting to me: The idea variability spectrum. (If you are interested to learn more about this and other related topics I can highly recommend this video with professor Barnett.)

The idea variability spectrum basically states that the farther from the neat norm-idea (basically incremental innovations) your innovation system span, the greater the variability of ideas. And the greater the variability of ideas, the greater the chance of those ideas being really bad or exceptionally good. Or perhaps better put in reverse order: The worst and the best of the ideas allowed to exist in your idea spectrum constitute the outer boundaries of the spectrum, as illustrated below. The only problem is, you can only tell after the fact which ideas were bonkers and which were brilliant in your portfolio.

So, let’s celebrate failure after all!?

 Not so fast… If you really need something other than success to celebrate, you should celebrate variability and learning. Because there is something very true about this mental model of different ideas inside a system. What’s really great about this model is that it’s telling us that:

“The only way to allow for great ideas to flourish is to also allow for really bad ideas to exist, and to not judge ideas as one or the other until after the fact.”

Unfortunately, this timeless insight has been reduced to “celebrate failure” in far too many organizations. Let’s explore why this is bad and what to do instead.

The best way to understand how the idea variability spectrum works is perhaps to explore a few possible shapes that are typical for different types of organizations. Let’s first consider a classic R&D-driven organization with a long legacy of successful incremental innovation, like Gillette or Tetra Pak. Their strategies (or cultures) are not founded on finding breakthrough technologies once or twice a decade, but rather (and only) incremental improvements maybe once or twice a year. These organization’s idea variability spectrum looks something like this:

Note that this spectrum is shaped by design, not incompetence. However, firms adopting this type of innovation strategy risk getting disrupted by other organizations taking advantage of new technologies or business models. It’s when companies aim to go from this narrow, neat idea variation spectrum to a broader one that, despite good intentions and flawless system-design, things can go very wrong… But before exploring why and how, let’s instead consider organizations that opt to only explore the edges of the spectrum, such as most tech startups, Alphabet’s X or DARPA.

What’s going on here? Basically, the organization has decided to put a 10X improvement prerequisite on everything it does. They’ve opted to not even consider incremental innovations, but only bet on ideas that will turn out to be either bonkers or brilliant. Makes sense.

Now, let’s go back to the organizations that want to go from a strong exploit strategy and culture, with a very narrow idea variability spectrum, to a more explorative strategy with a much broader spectrum. Why is it, that even though such initiatives may be super well designed and implemented, with top management support, resources allocated, organizational charts redrawn and innovation management systems set up, they sometimes (I’m tempted to write “usually”) fail to generate any meaningful outcomes?

Spectrum distortions may be the answer

Recent studies have made me believe that we may be fooled by distortions that exists in some contexts but are missing in others. For example, consider the phenomenal genius distortions created by a process designed and tested by Bocconi University, completely outcompeting other popular approaches such as LEAN Startup and Design Thinking. Or, the innovation success ratios associated with certain people (since their processes are different) that have formed startup studios around their own idea spectrum distortions.

I would argue that such distortions must exist within an organization for it to be truly successful in its exploration activities. It is equally true that a broad idea variety spectrum must exist in the organization, but the latter can be created without the former ever showing up to the party. In such a (very common) situation we may see all the KPIs moving in the right direction: The number of ideas increases, experimentation is also on the rise, failures are celebrated… But the brilliant ideas never come.

This is because any group of people can come up with crazy ideas, and they will if you incentivize them to. But only some can come up with truly brilliant ideas. In both cases, only a broad idea variability spectrum will allow for execution. Therefore, if we measure the output of the former and assume it to be causal (or even just inherently correlated to) the latter, we are going to fool ourselves in most cases.

What’s so interesting with this recent research, particularly the Bocconi studies, is that people can be trained to become better at ideation, idea refinement and idea selection. It’s also a theory explaining why some people like Bill Gross and Elon Musk enjoy such a disproportionate level of success from their own ideas, compared to much larger innovation systems. They are the distortions in the idea variety spectrum, while many innovation systems lack the presence of such forces.

What to do?

If your company runs a well setup and managed innovation system that allow for a broad idea variation spectrum, but still underperform, here’s my advice:

1. Assess your people. Do you have individuals in your organization capable of professionally generating many potentially brilliant ideas? And now don’t do this: “our people are brilliant engineers that…” Don’t generalize or romanticize. Don’t answer this question with pride. Instead, go and find one or two colleagues that can actually, materially prove you have these distortions present in your organization.
2. If you find out that the people in your organization don’t have any such traits, first of all don’t blame them! You probably hired them to build and run an exploitation-machine in the first place. This is important for most managers to learn: Anyone can turn out to be a hidden innovation genius. But not everyone can become one. So, don’t start some initiative to turn all your people into innovators Monday morning, that will backfire.
3. Short-term fix: Identify external distortion fields that would be amazing if you had them inhouse, and ask (read pay) for a contribution. This could be an innovation consulting firm, an individual, a particular startup (NOT just any startup), a creative agency, or any other organization or person that you can imagine would create a lasting impression on your business. Like, having Elon Musk in the office for a week.
4. Long-term fix: You may have to accept the fact that your organization doesn’t employ (and is not yet attractive to) people that have the innate ability to become distortion fields in the idea variation spectrum. But that’s ok. For this type of situation, the DARPA model is a good way to start. Because you are very likely to have super-awesome program managers and analysts in your firm, that can follow a proven model for breakthrough innovation where virtually all execution is done with external partners.
5. Ultimate fix: Set up your own venture studio with incentives in play to attract people that can create long-lasting distortion fields. Build it using the principles of Scientific Entrepreneurship as described by Bocconi University. This is an explorative environment that can co-exist with your exploitative organization, in an ambidextrous way, and provide wonderful growth opportunities to employees and your firm while still being an autonomous entity. Ping me if you need help making it happen. :)