The Failed Experiment That Made My Career

The experiment had not gone well. I had not got the answer that I was hoping for. 

Now I was in my boss’s office on a teleconference with our customers, bracing myself for the response to the bad news I had just given them.

On our last call we had talked about a problem with the process that we were co-developing. In a key step, about 5% of the particles that we were engineering were breaking down into smaller particles. These “fines” would cause all sorts of problems in later process stages and the final application. It was critical to figure out how to eradicate them. 

This was difficult. We could think of lots of ways that these fines might be made. And when we brainstormed all the things that could have an effect on fines generation it was a long list: temperature, particle concentration, heating rate, pH, surfactant concentration.

I had just been on a course on design of experiments where I learned how you could systematically vary many factors in an experiment to work out how to optimise a process or system. Looking for a solution to the fines problem seemed like a great opportunity to put my learning into practice. My boss and the customer team agreed.

I talked with my colleagues in the lab to get their experience and input on how to run the experiment. What would be a sensible range for varying each factor? It was worth spending time thinking hard about these things as each trial or “run” of the process meant 2 days of solid lab work. When we were happy that we had covered all the important practical points, I entered the factors and ranges into JMP software to get my experimental design. This plan told me how to set each factor for each run.

Despite many years training as a PhD chemist, meticulous lab work has never been my strong point. But I was very happy with how this experiment went. At the end of each run I was able to accurately quantify the amount of fines that had been generated. In fact, after completing the full experiment I saw that the results were remarkably consistent. That is when I started to get worried.

The maximum amount of fines that were acceptable was 2%. But in every run of my experiment the result was more than 3%. That might be okay if the results could at least tell us which factors were important and how we could use some untested factor combination to minimise fines. However, even though the factor settings were different for every run, %fines was pretty much the same each time.

The day before our customer call, I analysed the results in JMP, hoping that this wonderful software would provide some magical solution. But there is no magic to design of experiments. Just good science. 

This visual was the headline result that I presented to our customers:

Me: “This profiler plot shows how each factor on the X-axes affects our response, %fines, on the Y-axis. It is showing that none of the factors that we varied have a significant effect. The lines are all basically flat. Unfortunately there is no combination of factor settings, within the ranges that we experimented, that will gives us less than 2% fines.”

Customer: “This is quite… beautiful work, Phil. Thank you.”

I will never forget the look of surprised delight on my boss’s face!

Our customers did not think the experiment was a failure. They were right, of course. In minimum time and with minimum ambiguity we had been able to conclude that there was no solution where we had been looking. We could have wasted months chasing red herrings with the usual trial-and-error approach. Instead we could move on as a team and start thinking about where else we needed to look to find a solution. 

We did eventually fix the problem and the project was a success. Our head of development held up this work as an example to everyone else in our next big department meeting.

I never looked back. I carried out more designed experiments myself and found solutions that I would never have found otherwise. Then I started helping my colleagues to do the same. We had some big successes together.  

Soon I was working full-time advising people in different departments in the company on design of experiments and other data analytics tools. This led me to the job I have now at JMP, working with scientists and engineers around the world to help them understand how these methods can transform the way they work.

Design of experiments has given me a rich and rewarding career. This is why I am so passionate about sharing the value with other scientist and engineers. I am excited to announce that I will be delivering a free training course through Chemistry World, the magazine of the Royal Society of Chemistry. The interest has been incredible.

Register very soon if you want to attend this free training.