How a test and learn approach can help you...

In recent weeks, I found the topic of 'test and learn' surface in a few conversations with a client I’m working with. So much so I thought I would write about it to highlight some key findings and the value that this practice brings.

Simply put, test and learn is recognising a problem that needs to be addressed, considering ways in which it could be solved, testing them out, measuring the results and then amending and repeating the experiment until you learn how to solve the problem. The disadvantage of this approach is it can be expensive. Test and learn is particularly effective where understanding customer feedback fast is a high priority and tests can be launched quickly and cheaply. We commonly see test and learn applied in areas such as e-commerce and software development.

Ultimately, test and learn is a practice which if executed effectively allows us to better understand what products and services our customers want and how we can effectively deliver value to them.

Of course, with most things around principles and mindset, it's not something you can grasp and be effective at from the get-go. It takes time to apply and requires a mindset shift to continually try something different and be able to assess the impact of it. A culture shift is an even bigger feat to overcome, but as with most things when you wish to increase productivity, deliver great products to your customer, and create a happy workforce, small steps towards this practice is the key to success.

Mistakes are helpful. Learning from your mistakes has been studied extensively and findings by Hays, Kornell and Bjork suggest that memory improves if people try and fail to get a correct answer but are close to the mark. Trial and error helps learning be more effective than not making any mistakes, so long as the error is close to the correct answer, according to recent findings by Baycrest researchers. And another study by Potts and Shanks has suggested that making an error, even a guess, leads to significantly better memory of the correct information if feedback is provided after the initial response.

If you think about it, as children we are always testing and learning. This is the quickest way to learn by trying something (a task, a spelling test) followed by how did you get on? As adults, we sometimes feel that we should know it all and that everything we do should be done perfectly first time. This unfortunately is not reality, but adopting a culture where failures are transparent is daunting for any individual or team to come to grips with. If teams had the space to experiment, share their failures, discuss what improvements could be made next time, then you're creating this powerful safe space in which you can accelerate learning. Building trust with teams is critical for this practice to thrive and is the difference between a fixed mindset to a growth one.

People with a fixed mindset believe that they were born with a fixed level of talent and ability that cannot change, rather than working to develop and improve them. They believe that talent alone leads to success, and effort is not required. With a growth mindset, people believe their abilities and intelligence can be developed with effort, learning, and persistence. Their abilities are simply a starting point for their potential. In order to develop a growth mindset, you have to be willing and committed to changing your beliefs about what you are able to do.

Growth mindset is very much linked to an intrinsic motivation trait called Mastery. A topic I may write about in the future, but it's an idea that you, your team and your company are relentlessly looking to learn and improve how you deliver projects and products. In my experience helping teams develop software products, I've found that there are mainly two cycles of test and learn; one around the work that we deliver and another around the way that we deliver that work. The use of feedback loops validates assumptions about our problem-solution space to ensure customer needs are being met. We also use them around our system of work so that we learn what works (that is, the practices and principles that we should amplify) and what does not work (those that we should get rid of or do differently). This helps us to deliver value more effectively.

For the two cycles to be effective I've learnt that all team members need to see the value of this practice. There are a few reasons why clients see failure and the practice of experimenting as a negative thing:

1. They have not witnessed a successful outcome of the practice in the work that they’re doing for themselves.
2. There's a fear culture in that sharing about time spent on a failure is a sign of weakness.
3. Teams are so focussed on only meeting deadlines and commitments that spending time on process improvements or creating an exploratory task isn't valued as highly.
4. Some may like to discuss issues and areas of improvement but find implementing change more daunting due to a lack of stakeholder support.

So, what's the answer and how can anyone who's looking to build a 'test and learn' culture get started?

Here are a few things I’m going to share with you to try:

1. Get buy in from your stakeholders and sponsor. Explain to them the challenges you’re currently experiencing in problem solving or where inefficiencies exist. At this point, you’re recommending that you simply try something different.
2. Explain the advantages of test and learn in relation to the big picture. Give every team member, stakeholder and sponsor this broader information and use evidence to show the validity of test and learn including case studies.
3. Prepare for ongoing experimenting. Be realistic with expectations and it’s important to remember that learning comes from mistakes just as much as successes in testing.
4. Involve everyone in all aspects of the process to create a sense of ownership and responsibility. Create a safe space for team members to ‘fail’ and share experiences. The larger the safe space the more open team members will be outside of the team therefore the learned value is shared further.
5. Share the journey you've taken with others, share the difficult moments, the challenges you faced and the outcomes (good and bad). Demonstrate to others the benefits of this practice and how you can support them on their journey.

Ultimately the 'test and learn' goal is to take small, smart steps which allow you to pause and reflect on what has been achieved, and to build your learnings into the next cycle. It goes without saying that part of this process will be failure. As you edge along your journey, you may slip on ice even if you were stepping cautiously. It’s best to consider what just happened and carefully continue along your path.

The test and learn approach enable businesses to make progress using data and effectively deliver products and services to their customers. And what business wouldn’t want to excel in this?