AutoML - first glance
Raja Saurabh Tiwari
Vice President @ Citi | Java , Cloud, ML Solutions | Gen AI enthusiast | Wildlife Photography
"Machine Learning and AI attempts to automate manual work...
AutoML attempts to automate Machine Learning process."
Machine Learning appears very fascinating to most of the new comes, but it involves a lot of dirty job inherent in it.
For example if we talk from very high level (1000ft above) the machine learning process would have following steps
- Collecting the data/information from various sources
- EDA , cleaning the data, understanding the data
- Selecting the right family of model
- Fitting model, predicting and testing it
- Validating your model
- And finally deploying the model as a complete solution
The ML aspirants/engineers find the job of making model, predictions and further steps very fascinating. But in my opinion that's smaller chunk of the real work one has to do. The main work lies in analyzing the data i.e. gathering, cleaning and understanding the data. Which eats up majority of the time and this requires real data science skills.
Many of the steps in process are iterative and you wouldn't even know if the approach you have chosen will work unless you have wasted lot of your time already.
These tasks are very complex for the non-ML experts and appear very daunting for beginners.
So that gives an 'opportunity' to have one OTS easy to use software/library which would help any beginner to achieve the goal in few lines of code. The AutoML comes into the rescue here.
AutoML helps in building a complete pipeline from raw data to deployable machine learning models in few lines of the code.
So essentially what you see in the black box is all taken care by the AutoML which is the heart of the machine learning. In some tools optionally you should be able to provide Optimization metrics also along with some constraints.
It also gives you ,
- Simplicity : You don’t have to go deep into the model complexity and algorithm details.
- OTS solution : It gives you OTS solution which is easy to code and deploy
- Robustness : Provide the raw data, and that's it.
- Fault tolerance : Some tool provide you ability to resume if interrupted.
All the big players like Google and Amazon have already started providing AutoML solutions as
Cloud AutoML
Sagemaker, AutoGluon
Image Courtesy: Google
For me, AutoML has recently emerged and still under evaluation. Organizations have started using it but at a very low scale right now. This has to mature in a 'true machine learning' way by going through data and experience. So let's wait and watch.
Raja Saurabh Tiwari