The Power of Experimentation

In the corporate world making wrong decisions can lead to negative consequences:

• Loss of customers
• Damage to relationships
• Loss of credibility
• Team member attrition
• Demotion or job loss

In fact, Jeff Bezos the CEO of Amazon is reported as saying that the trait he looks for most when promoting someone into leadership is “I want people who are right most of the time”. This type of environment can cause leaders to spend too much time analyzing and theorizing the effect a change might have. It can also prevent them from pivoting away from changes that are not going well and persevering too long supporting a change that doesn’t accomplish the desired results. In my experience a remedy to this is creating an environment where experimentation is understood and is accepted.

With Scrum we do this by limiting the duration of the experiment to one month or less in a Sprint. We define the result we are looking for as a valuable, useable, potentially releasable Product Increment that meets the definition of “Done”. It is only when we release the Product Increment our hypothesis of value is validated. We inspect the results of our increment in the Sprint Review which promotes a dialogue between the customers of the experiment with the possibility of influencing future experiments selected from the Product Backlog. The team also inspects itself in the Sprint Retrospective and determines how it can get better at running experiments and includes a new experiment that applies specifically to themselves in the next Sprint. 

A similar model can be applied for making difficult decisions in highly ambiguous situations where best practices are not intuitively known and able to be easily applied and accepted.

• Understand the problem you are trying to solve (easier said than done many times…you can use “5 Whys to help you gain clarity).
• Create an experiment to address the problem…limiting the scope appropriately to contain the risk but also significant enough where you can inspect a result.
• Determine the benefits you are hoping to achieve.
• Predict potential drawbacks of the change.
• Determine the duration of the experiment with a preference to shorter time frames to get feedback sooner.
• Evaluate the results of the experiment and make an adaptation decision:

1. Grow the experiment (high benefits, low drawbacks) – Enjoy these when they happen.
2. Change the experiment (high benefits, high drawbacks or low benefits, low drawbacks) – Continue inspecting and adapting to see how you can improve your results.
3. Question the experiment (low benefits, high drawbacks) – Did you understand the problem? Don’t give up…but challenge your assumptions.

If we can make the above as transparent as possible in the organization with the understanding that our desire is always to learn, improve, and provide more value to our customers we can be “right” even when our experiment didn’t go as we had hoped.

While this article was working its way through my brain to the keyboard I found an incredibly useful and insightful article by Esther Derby called Change Artist Super Powers: Experimentation. In the article she has ten great questions and some great advice on this subject. I would recommend you check out her article and then get to work performing some experiments!