Can the Magic of AI Replace Science?
I believe in the scientific method. It has done so much for us. I want to assure you that smart people recognise that it is still key to our innovation and progress.
In my work I regularly talk to scientists, engineers and leaders in the process industries (biotech, semiconductor, pharma and chemicals…). I see so much hype and confusion about the impact and opportunity for Artifical Intelligence (AI) and Machine Learning (ML).
Using data in the scientific method
I try to bring clarity about how data can be used in the scientific method to solve technical problems and answer business questions. You can think about how it generally works like this:
- What is the technical problem or business question?
- What model of our system or process will give us the understanding to answer these questions?
- What data is needed to develop this model?
From here we collect the data; through new experiments, by exploring existing data, or a combination of the two. Then we use data analytics (statistics, ML) to create models and answer our original questions. Most of the time we cycle back with new questions or to refine our models and provide more useful answers.
Upside down thinking
The problem I often see is when companies approach this the wrong way round:
- What data do we have?
- What understanding of our systems or processes can we get from models of this data?
- Can we now solve any technical problems or answer any business questions?
The thinking behind this is something like this...
Data is the new oil. Logically if we’ve got lots of data, we’ve got lots of untapped value. All we need to do is get it together into a big lake and we will make loads of money. We will measure our success in terabytes, petabytes and exabytes. We can just use the magic of AI to answer any question that we would ever have. Why would we pay scientists to keep collecting more data?
Many years and many $Ms later we have our data lake and nobody uses it. Turns out the data is not all that useful for answering the important questions. Plus nobody thought about what tools would be needed to do something with this data.
What the smart people do
Now I want to share a valuable lesson about data and science from a surprising source.
Netflix is a company that has done really well through digital transformation. They are very open about how they use data analytics to create business value. It doesn’t happen through some magical application of predictive analytics. The scientific method is key:
…experimentation may be thought of as being superior to most ML approaches that are based on observational data. We do spend a significant amount of effort in researching and building ML models and algorithms. Carefully exploiting patterns in observed data is powerful for making predictions and also in reaffirming hypotheses, but it’s even more powerful to run experiments to get at causation.[1]
Just passively collecting data is not enough. Even more so in the process industries where there will always be technical and business questions that can’t be answered with existing data. If you want to reduce costs and get your products to market faster you will find you often also need the “small data†analytics toolkit, including Design of Experiments (DoE). DoE is how you build models and understanding that you need to answer your business and technical questions while minimising your spend on experimentation.
Data analytic tools in the hands of scientists and engineers
To get the most value from data in industry I believe that you should put scientists and engineers at the heart of this. You should enable them by putting the data analytics tools in their hands. They have a unique understanding of your technology that can’t be replaced.
Experimentation and the scientific method will always be central to innovation in industry.
If you are interested in how scientists and engineers can be enabled to apply data analytics to solve problems in industry, have a look at this free online course created by JMP to give people a grounding in the most important tools of statistics.
[1] A/B Testing and Beyond: Improving the Netflix Streaming Experience with Experimentation and Data Science, Nirmal Govind
Project Director & Group Manager
5 年A friend of mine says that in our field (materials science) ML can only interpolate within the dataset we have, and this precludes innovation. To innovate, you need physics. AI won’t replace theory.
Adjunct torturer (I teach math and stats) and push boundaries that should never be.
5 å¹´Sorry for getting to this so late. I had a customer give me all their data for a particular issue they had with Warranty claims. They had some ideas about what to do. But, setting up a DOE would have been impossible. The "event" occurred about 0.03% of the time, or less. I ended up using CART models to show the dept head. The only important factors were covariates.... and only at certain settings. Btw, the dept head was very impressed with how quickly I could give him answers using JMP.
Director, Process Insight Consulting - Business Improvement Specialists
5 å¹´Good article Phil
Enterprise Account Manager at AWS
5 å¹´Great article Phil. I regularly see AI and Machine Learning being peddled as "the solution". The assumption that AI and Machine Learning is the panacea to solving complex scientific and engineering problems is a costly and potentially dangerous mistake to make. I'd like to hear of more people challenging this assumption, so that they are not misled.
DOE & Data Analytics Evangelist | Nervously excited about Digital Future of Science, Engineering, R&D, Manufacturing | Medium-pace runner and road cyclist
5 å¹´This white paper by Roger Hoerl should be of interest if you found this article useful: https://www.jmp.com/en_gb/whitepapers/jmp/integration-of-big-data-analytics-holistic-approach.html