Why Experiment?

I love listening to scientists and engineers talking about the value of adopting smarter strategies to experiment. But we rarely talk about why we experiment in the first place. It seems obvious but I think there is a lot of confusion about the reason for experiments. I think a lot of this is down to how we are educated. It took me quite a long time to really grasp that we experiment to gain new understanding.

Like a lot of other scientists, I did many “experiments” in my school and undergraduate studies. I would follow a set of instructions or a recipe and that would (mostly) lead me to an outcome demonstrating some important learning. It seemed science was about accumulating knowledge from teachers and books. But was I really experimenting? Was I really learning about science?

As an undergrad student on my placement year I went into an industrial chemical development lab for the first time. And it wasn’t what I was expecting. Where were all the whiteboards with chemical structures and curly arrows? For the non-chemists reading this I should explain... 

Up to that point in my career it seemed that the most important tool to a chemist was the curly arrow. In classes we used them to show how electrons move within and between molecules and therefore how you can expect chemical reactions to proceed. I was fairly sure that this is what all the scientists were doing in chemical companies. 

In industry I quickly found out that the reality of process development is that many reactions and chemical processes are too messy or complex to be figured out using theoretical chemistry understanding alone.

To be fair, the scientists in that lab were using their understanding of chemical mechanisms to explain and predict the results of their reactions. However, that was not what they spent most of their time doing. Instead they were carrying out preparations or “preps” of new substances and materials. It was confusing that they didn’t call them experiments. Between each attempted prep they would make slight changes to the recipe, with the aim of improving outcomes like yield, purity and performance in application tests. I followed this approach over the course of the 14 months that I was there and produced various powders and sludges. Some of which showed some promise in application. 

Calling them “preps” was a real misnomer. Which is why, when I returned to industry after a PhD, I was still confused about what I was doing in the lab as a process development chemist. I thought my job was to prepare material. If I made “good” material, I went home happy. If I made “bad” material, I went home miserable. It was a process of trial and error to move from “bad” to “good”. With each prep I would make a slight change to the recipe in the hope that it would make it better. I would typically end up doing a lot of preps before I got to something that was good enough to go into commercial manufacture. And each prep could take several days to complete.

Slowly I came to a realisation. A realisation that my job was to make process understanding. Part of that realisation came from seeing this kind of visual: 

This simple visual communicates our process understanding: what we need to control and how we need to control it to ensure the best quality outcome. In this case you can see that we need to control catalyst amount, temperature and time in this process to get the best yield. The feed rate and agitation rate are not so important. You can see that there is a sweet spot that we should use to give us the highest yield. And you can see how deviating from those settings will affect yield. This is the understanding we need to efficiently and consistently make high quality product. This is what we need to get from our experiments.

In industrial R&D and manufacturing we are trying to solve problems. To solve these problems we need to understand how the process or system works. We need to to understand the effect of changing amounts, ingredients or processing conditions. We need to experiment because theory is not enough to tell us how to make our technology work. We are making products that nobody has made before so there is no recipe book. We need to experiment to gain the process understanding we need to get our products to market.

If you want to hear how successful organisations get better products to market, faster with smart trial and error strategies, watch this from our recent Statistically Speaking seminar: