Organisations That Experiment

This is very early stage #WorkingOutLoud as i conduct the initial research for the book on ‘Experimenting Organisations’. So early stage that it may barely be worth reading, but i am finding it useful to rapidly iterate the structure of my thinking in service of finding something that ‘works’. This illustration is the fourth draft, the first one worth sharing, and illustrates an idealised ‘experimenting’ process. Maybe… i’ll talk you through the narrative, with the caveat that i know how incomplete it is.

I start with the MOTIVATION to experiment: are we experimenting around pain points, or native curiosity. Need or desire. I feel that an Organisation that experiments well will understand it’s own motivation, and potentially ensure that it experiments broadly, as well as in depth.

Related to this, but shown separately for now, is the CONTEXT of the experiment: part of the flow of work, as an exceptional activity, within a dedicated unit, or distributed broadly around the Organisation. This relates to cultural factors and ties back to motivation.

Finally, i added BIAS as a conditional factor: do we have an understanding/narrative around how our current context and understanding biases what we seek to explore. E.g. a bank, with expertise in current regulation and shared product categories may need to articulate it’s own bias as to the shape of future innovation and market.

In my conversation so far, DEFINITION is one of the variable factors: a small number of Organisations formally define experiments in a standardised structure, whilst a larger number devolve it, or are happy with ad hoc structure, at least in the early stages.

As a side note here: at which point we impose structure and oversight is again a broad variable. Some impose it from the start, others when it starts to impact budget or scale, and yet others never standardise structure or even reporting.

I’ve added two other elements here: LEARNING OUTCOMES, and GOVERNANCE. Learning Outcomes i am less sure about, because it restricts us to purposeful learning (more so than just ‘wading in’), but in an Organisational context, it feels important. Some of the more structured organisations i have spoken to tend to formally define learning objectives, and none reported this as arduous. Knowing what you hope to learn: arguably this box could have related to HYPOTHESIS, but it’s worth noting that (contrary to a more scientific approach) some Organisations at least never formalise a hypothesis, sometimes settling instead for Design Questions alone – or Objectives they hope to explore. Pedants can argue as to whether this truly counts as an experiment, but i am tending towards a view that if it is inductive, or deductive, reasoning at heart, it counts.

ACTIVITY is self explanatory: and to state the obvious, Organisations that experiment are pretty good at actually getting into action! It may be worth mentioning that there are plenty of Organisations that never actually get to the start line, or never do so at scale. They run small numbers of high profile experiments, but largely lack capability to rapidly experiment dozens of times. I suspect because culture constrains it.

I have found interesting in the interviews that some Organisations have very clear approaches to ESCALATION. Not simply when things go wrong, but often when they feel analysis can be stronger if a broader perspective is taken, or that learning will be more useful if carried higher and spread more widely.

Similarly, defining explicit STOP CONDITIONS may be a sign of maturity around experimentation: immature approaches see Organisations scaring themselves when the unexpected occurs – almost a defining feature of experimentation at scale and speed!

SENSE MAKING and ANALYSIS are pretty obvious, although again there is clear differentiation between those Organisations that see these as specific, or indeed certifiable, skills, and those that rely on intuition alone.

Again, a small number of Organisations describe structured approaches to sharing: less mature ones take divergent and intuitive approaches. So potentially both types of Organisation learn from experiments, but in one the learning has the potential to be global/systemic, whilst in the other it is more likely to remain tacit and tribal.

Finally: INTERCONNECTION is about where the capability sits to conduct the meta analysis across a range of experiments. Almost a story of how we find the needle in the haystack: time, expertise, resource, are all important here. LOOPING is simply a matter of good practice; every time you complete an experiment, your last sentence of the story should be ‘and this is where i will look next’.

I will continue to iterate and share this work as part of #WorkingOutLoud. As a reminder, the book on ‘Organisations that Experiment’ will be based around case studies, alongside whatever frameworks i can conjure up. I intend to do the primary research through this year, and publish next summer.