Does experimentation inhibit or encourage innovation?

TLDR; Experimentation has its place. Intuition has its place. There is inherent bias with validating the new be that a pre-conditioned way of thinking, fear or timing which is what I want to explore…

"Don’t experiment when you should think; don’t think when you should experiment" Jon Bentley

This is going to sound counter-intuitive coming from someone who has built his career up evangelising experimentation. And as someone who considers creativity one of his core values. But I'm not sure the two work that well hand in hand.

I genuinely feel that experimenting can restrict creativity and innovation just as much as it fosters it.

What?!

I've set out exploring the concepts of intuition, innovation, disruption and how the methodology of split testing works alongside them, if at all, and found that the majority of people believe that yes "experimentation encourages innovation".

How Innocent drinks experimented with their smoothies. Or how Dropbox experimented with their product and pricing offering. Or how Uber launched UberPool to influece Uber usage. The moon landing was a series of experiments was it not?

Heck, I even put a biased, small sample sized poll on Linkedin just last week and the results speak for themselves.

But what about the adverse?

What about Henry J Ford asking his customers what they would have wanted? Faster horses of course. Is that considered too radical an innovation that experimentation can't support such a concept because at some point you've just got to take the leap? Users don't know what they don't know right? Or maybe it's just a matter of timing.

Indeed, does experimentation corrode the purpose of intuition? Tim Berners-Lee didn't wake up one day and say "I'm going to test the concept of this interconnected web thing called the Internet" or iterate towards it. His boss even called it "vague" and took 4 years before it became non-proprietary.

Not even users know what they want.

"People have told us over and over again ,they don't want to rent their music...they don't want subscriptions" Steve Jobs in 2003.

When we listen to our customers, they can be often wrong. Just as we are; despited being professed 'experts' within our field.

More precisely, when we interpret or analyse what customers want, in theory they know no better. I'm not devaluing the need for research by the way - heck, even Jakob Nielsen says don't listen to your users.

I'm curious as to the role that research, which experimentation is a part of, plays in the innovation of new things. How can users know what they want when it's not been created or adopted yet? How do companies like Spotify innovate when the apparent research suggests customers 'don't want to rent their music'? Or how does Ford innovate with the car when the apparent research suggests that customers 'want faster horses'?

Or the adverse might happen by actually listening to your customers; when responding to a survey Walmart posed, customers asked for less clutter, Walmart did so and in the process lost $1.85bn in sales

Within innovation or new product development, I question, can feedback dilute the innovation of what's intended to be? The BBC didn't concede to negative feedback when they radically redesigned their homepage in 2015. It soon calmed down as they became more conditioned to the change itself (h/t to Tim Axon)

That's because we don't know what we don't know.

"What's really valuable in life is when you allow people to do different things. Things they otherwise wouldn't have contemplated or things that didn't simply occur to them" Rory Sutherland

When we come up with solutions, be they for experiments or not, we tinker the experience based on what our experience with the experience is. This is because of our pre-conditioning as well as functional fixedness, a cognitive bias that involves a tendency to see objects as only working in a particular way.

In other words, we are conditioned to variables or stimulus. i.e we only know what we know. In our world, a PDP has to look a certain way for a specific purpose, or filters must be located here and have these functions.

We use our past experiences to help solve the issues we face. This can make it difficult to see novel or creative ways of fixing current problems (Valee-Tourangeau; Einstellung defused Interactivity and mental set).

My hypothesis goes further beyond 'creating the same solutions to the same problems' because of our conditioned stimulus. And I propose, the more we homogenise solutions, the more users become desensitised to the effect of the 'new'. Meaning the homogenisation of solutions will impact user conversion (h/t) less and less over time. Making it even harder to innovate.

The new requires time and effort that we don't have

"The best, maybe the only real measure of innovation, is a change in human behaviour" Stewart Butterfield

From a user perspective, I question whether learning a new product or feature, because it carries a change in behaviour, carries both effort and risk; and therefore brings with it avoidance.

In that, as humans, we fear catastrophe. There are many more ways to die than there are to survive. We know we want something that's pretty good, but we definitely know that we don't want it to be terrible. Therefore, we are conditioned to safety in what we know, and in what we don't know. Well. We avoid because we fear.

I think we also avoid because we associate effort and time in learning the new. There's a bit to be said about McDonalds and Amazon here. In that, we want things fast. Like, super fast. There's even the coined term "McDonalidsation" of society, of which efficiency is the first principle, and, to my point earlier, predictability is the third. But learning something new comes at the cost of speed.

I remember testing Klarna for a beauty brand in 2016, less than 6 months after it "came to market" in the UK. We tried 9 different iterations of promoting it to clients and every single time it 'failed'. There was next to no uptake on a service that, in theory, should have been perfect for this brand. Our assumption as to why was as much about the communication of the innovative buy-now pay-later service, less so the product itself.

It's one thing to come up with a product or feature. It's a whole new thing - remarkably more difficult and often lesser discussed - to get people to adopt a new behaviour. Therefore, when it comes to experimentation, the hypothesis process should have as much, if not greater emphasis, on the communication of the product, as much as the product itself.

The more we understand about something, the more restricted our river of thinking.

Indeed, does greater understanding of something lead to an inability to innovate in that area? We are so conditioned to how a pricing within SaaS products should be, for example, does that inhibit our ability to innovate or disrupt that model?

There's certainly the argument to suggest that experiments validate and provide knowledge. I truly believe that, also. (note: it's something I'm going to unpack more in "can hypothesis limit creativity")

"Experiments are fact-finding missions that, over time, inch towards greater understanding" Andrew Stanton

But there's the counter argument to this. Having a, what Duncan Wardle says, "naive expert" in the room can create more than those with greater understanding. Someone who isn't an expert within the field and therefore does not have any preconceptions, nor bound by the limits of what is and isn't possible. It's how Lemonade insurance was born, now valued at over $2bn in just two years. It's how, in Duncan's case, when asked to draw a house you always draw a door, two windows and a triangle for a roof because you understand how houses should look.

I question the outcome of Andrews statement; can greater understanding limit the art of what's possible or remove preconceptions and therefore innovation?

Summary

I think there's a place for the methodology of experimentation when it comes to innovation.

In my opinion, experimentation supports innovation as its a method for validation and exploration; but it doesn't create it. There are inherent bias within validation, too, given users might not know any better or be ready for that innovation itself.

Which means there's a place for intuition.

That's not to say that you read the results of an AB test, feel in in your gut that it's wrong, and ignore the data. In fact the data can lead you to other areas of curiosity - think the creation of the Post It. But stepping outside our comfort zone, our preconceptions, our river of thinking, testing in different ways that are more than data-driven, understanding that a split tests and iterations of that are not just the only way to solicit feedback and understanding; are methods of creative thought that can really innovate.

It's a curious topic that needs more unpacking. In my next series of articles, I am going to dive a bit deeper into questions like:

• Does experimentation encourage creativity through validation?
• Can hypotheses limit creativity?
• Experimenting for iteration and experimenting for innovation
• How can we be more creative within our process of experimentation?

What do you think? Does experimentation inhibit innovation as the solutions and the users that interact with it are too preconditioned in their river of thinking? Is experimentation vital in the creativity process because it allows us to learn and understand relationships, therefore iterate to places we've never been before?

Thanks to Tim Axon for giving me guide rails, Mark Pybus for being Pensive Pybus, Oliver Palmer for telling me structure is everything, and Leon Andrews for being his empathetic self.