The Beginning matters most – How to effectively start your experimental journey.

When we get the task to conduct experiments to solve a business problem like bringing a product to market as fast as possible or finish a R&D project to help with production issues, we naturally tend to immediately dive into experimentation – either because we are curious or because of internal or external time pressure.

This can potentially lead to hick ups which can follow us our whole experimental process and can make our life harder. Let us try to find six high-level thoughts and questions you should ask yourself right at the beginning to avoid stress, redundancy, and inefficiency. Some of them might sound trivial to experienced experimenters. Congratulations, you probably already ran into some of these issues like me and many others before. Nevertheless, these are among others the most common obstacles when planning and conducting experiments. In case some of the questions resonate with your experimental obstacles use the key words to learn more about methodologies to tackle them.

1.????? Define Clear Objectives

This sounds natural but is often not clearly defined. What metric is going to be measured? How many metrics? Do they need to hit a target, need to be maximized or minimized? Is there a decreasing importance of the metrics? Can they be measured on a continuous scale, or do they have a categorical nature? To have these things defined is the first step in every experimental designing process.

?

2.????? Make sure the outcome is measurable

Is your measurement system capable of tracking important differences between your measurements? Measuring submillimeter sizes with a folding rule makes little sense. Often, however, problems with the measuring system are not so obvious. Some measuring systems themselves introduce variation in the measurement result, which is sometimes greater than the real differences between the samples. Having a sense for problems related to the measuring systems can save headaches later.

Key words: Measurement System Analysis; Variability Chart; Gauge R&R

?

3.????? Which parameters should be included in our experiments?

What parameters do we expect to have a high impact on our outputs? Domain knowledge is key here. The number and nature of parameters is directly influencing the amount of experiments we need to conduct and should also influence the choice of experimental design. But it's not just the parameters we study that matter. What about the parameters that we do not vary? Can we keep them constant? At what level? Do they fluctuate? How strong? Careful considerations here will result in a good system overview right at the start.

Key words: Cause and Effect Diagram

?

4.????? How your parameter space influences your results

Choosing the right ranges for the parameter of interest is one of the most important steps when designing experiments. Too narrow ranges carry the risk that the influence of the parameter will be lost in the noise, while too broad ranges can lead to unmeasurable results due to extreme conditions in the system (e.g. killing cells in reactor due to excessive heat or no change in a sample caused by too little pressure). Domain knowledge and data from previous experiments again are key here. If there is no prior knowledge it could be a good idea to investigate the assumed extreme parameter conditions in the low and high end of the parameter space with some scoping experiments to make sure that these combinations produce measurable results and are distinguishable.

Key words: Scoping Designs ?

?

5.????? Experimental budget

Our experimental budget determines how many experimental runs we can perform and how much knowledge we will be able to gather. Usually, time or costs are the restrictive factors here. The budget directly influences the optimal type of experimental design. We should always keep some runs for confirmation experiments of our prior results. Keep in mind that experimentation should be a sequential approach. It is perfectly fine to start with scoping experiments (like the ones mentioned above), continue with screening experimentation and then finish with experiments to find the settings of the lower number of parameters (flagged by the screening experiments) needed to optimize our outcomes. The first design often times is not the solution to our problem. Taking this into account and to start simple can save a lot of trouble later on. ???

Key words: Screening Design; Optimal Design; Definitive Screening Design; Design Augmentation

?

6.????? Considerations regarding noise

Ever tried to repeat the same settings of your experimental system for a couple of times? This can help determine the noise in our system. Does changing our parameters give us a bigger impact on the outcome than the noise? This can be included as scoping experiments as well. Noise exists in every process, but the amount of it gives us information about the probability of success of our experiments. How far do we have to define our parameter window to cut through the noise? Can we live with the natural variation when using the process later in production?

Key Words: Signal to Noise Ratio

?

Wrap-Up

As you can see there are several questions which might be good to ask ourselves at the beginning of every experimental journey. Especially when transitioning from the one-factor-at-a-time (OFAT) approach to Design of Experiments (DOE) methodology the search for the right experimental design often dominates our thinking but please remember the guidance here to increase your chances of gaining useful knowledge with which to solve your problem.

If you want to learn more about DOE, feel free to follow:

#DOEbyPhilKay on LinkedIn

or Victor Guiller on LinkedIn