10 steps to build and optimize a ML model

TLDR

Let’s take a look at the different steps to build a prediction model and go over the?what,?when,?why, and?how?people accomplish them.

Steps

Below are the steps required to solve a machine learning use case and to build a model.

1. Define the Objective
2. Data Gathering
3. Data Cleaning
4. Exploratory Data Analysis (EDA)
5. Feature Engineering
6. Feature Selection
7. Model Building
8. Model Evaluation
9. Model Optimization
10. Conclusion

Step 1: Define the objective

Source: Pixabay

What’s the objective?

Deciding a use case you want to predict or know more about.

When is the objective defined?

Objective is the first step which is decided based on business requirements.

Why is it necessary to set an objective?

Defining the objective sheds light on?what kind of data should be gathered. It also helps us in?judging what kind of observations are important?while doing exploratory data analysis.

How to define an objective?

Objective should be clear and precise. Therefore, to define a clear objective we need to follow few steps like:

• Understand the business (Eg: Grocery store)
• Identify the problem (Eg: Less Profits)
• List out all the possible solutions to solve the problem(Eg: By increasing sales or by reducing manufacturing costs or by managing inventory etc.)
• Decide on one solution (Eg: managing inventory, we can come to this conclusion by talking to the respective business people back and forth.

By following the above steps, we’ve clearly defined that the objective is to build a model?to manage inventory?in order to?increase?store?profits.

Step 2: Data Gathering

Source: Pexels

What’s Data Gathering?

Data Gathering is nothing but collecting the data required as per the defined objective.

When do we gather data?

Once the objective is defined, we will collect data.

Why is Data Gathering necessary?

Without past data, we cannot predict the future, hence Data Gathering is necessary. In general a dataset is created by?gathering data from various resources?based on the objective. One of the reasons for gathering data from multiple resources is to get more accurate results i.e.,”The more the data, the more accurate the results will be”.

How is Data Gathering done?

Data can be collected in one of the following ways mentioned below:

• API’s (like Google, Amazon, Twitter, New York Times etc.)
• Databases (like AWS, GCP etc.)
• Open source (Kaggle/UCI Machine Learning Repositories etc.)
• Web Scraping (Not recommended, as often it is considered as illegal)

“The order of?Defining the objective?and?Data gathering?steps can be changed. Sometimes we will have the data handy and we need to define the objective later and sometimes we will decide the objective first and then we will gather data.”

Step 3: Data Cleaning

Source: Pixabay

What’s Data Cleaning?

Data cleaning is the process of removing, modifying or formatting data that is incorrect, irrelevant or duplicated.

When to clean the data?

Once we have the dataset ready, we will clean the data.

Why is data cleaning necessary?

Data Cleaning helps in preparing the data for Exploratory Data Analysis.

How to do Data Cleaning?

We use libraries like?Pandas,?Numpy?to do Data Cleaning and apply the following?key steps to determine if we need to clean?the dataset.

1. Check how many rows and columns are in the dataset.

2. Look for?duplicate?features by going through the meta info provided.

3. Identify?Numerical?and?Categorical?features in the gathered data and check if?formatting?is required or not.

“Formatting can be something like changing data types of the features, correcting the typos or removing the special characters from the data if there are any.”

“If you are working with real time data, then it’s recommended to save the cleaned dataset in the cloud databases before the next steps.”

Step 4: Exploratory Data Analysis (EDA)

Source: Pixabay

What’s EDA?

In simple terms, EDA is nothing but understanding and analyzing the data by using various Statistical Measures (like mean, median) and Visualization Techniques(like Univariate Analysis, Bivariate Analysis etc.).

When to perform EDA?

After the data cleaning stage. Once the data is cleaned, we perform EDA on cleaned data.

Why is EDA necessary?

Exploratory Data Analysis is considered as the?fundamental?and?crucial step?in solving any Machine Learning use case as it helps us to identify trends, or patterns?in the data.

How to perform EDA?

There are Python libraries like?Pandas,?Numpy,?Statsmodels,?Matplotlib,?Seaborn,?Plotly?etc, to perform Exploratory Data Analysis.

While doing EDA, some of the basic common questions we ask are:

1. What are the independent and dependent features/labels in the collected data?

2. Is the selected label/dependent feature Categorical or Numerical?

3. Are there any missing values in the features/variables?

4. What are the summary statistics (like mean etc.) for Numerical features?

5. What are the summary statistics (like mode etc.) for Categorical features?

6. Are the features/variables normally distributed or skewed?

7. Are there any outliers in the features/variables?

8. Which independent features are correlated with the dependent feature?

9. Is there any correlation between the independent features?

“So, we will try to understand the data by finding answers to the above questions both?Visually?(by plotting graphs) and?Statistically?(hypothesis testing like normality tests).”

“When we are dealing with larger datasets, then it’s a bit difficult to get more insights?from?the?data. Hence, at this stage we sometimes use?Unsupervised learning techniques?like?Clustering?to identify hidden groups/clusters in the data which thereby helps us in understanding the data?more.”

Step 5: Feature Engineering

Source: Pixabay

What’s Feature Engineering?

A?feature?refers to a?column?in a dataset, while?engineering?can be?manipulating, transforming, or constructing, together they’re known as?Feature Engineering. Simply put, Feature Engineering is nothing but transforming existing features or constructing new features.

When to do Feature Engineering?

Feature Engineering is done immediately after Exploratory Data Analysis (EDA)

Why is Feature Engineering necessary?

Feature Engineering transforms the?raw data/features?into?features?which are?suitable for machine learning algorithms. This step is necessary because feature engineering further helps in improving machine learning?model’s performance and accuracy.

Algorithm: Algorithms are mathematical procedures applied on a given data.

Model: Outcome of a machine learning algorithm is a generalized equation for the given data and this generalized equation is called a?model.

How to do Feature Engineering ?

We use libraries like?Pandas,?Numpy,?Scikit-learn?to do Feature Engineering.

Feature Engineering techniques include:

1. Handling Missing Values

2. Handling Skewness

3. Treating Outliers

4. Encoding

5. Handling Imbalanced data

6. Scaling down the features

7. Creating new features from the existing features

Step 6: Feature Selection

Source: Pixabay

What’s Feature Selection?

Feature Selection is the process of selecting the best set of independent features or columns that are required to train a machine learning algorithm.

When to do Feature Selection?

Feature Selection is performed right after the feature engineering step.

Why is Feature Selection necessary?

Feature Selection is necessary for the following reasons:

• Improves Machine Learning Model performance.
• Reduces training time of machine learning algorithms.
• Improves the generalization of the model.

How to do Feature Selection?

We use Python libraries like?Statsmodels?or?Scikit-learn?to do feature selection.

Each of the following methods can be used for selecting the best independent features:

1. Filter methods

2. Wrapper methods

3. Embedded or intrinsic methods

“If the number of selected input features are very large (probably greater than the number of rows/records in the dataset), then we can use Unsupervised learning techniques like?Dimensionality Reduction?at this stage to reduce the total number of inputs to the model.”

Step 7: Model Building

Source: Pixabay

What’s Model Building?

Building a machine learning model is about coming up with a generalized equation for data using machine learning algorithms.

“Machine learning algorithms are not only used to build models but sometimes they are also used for?filling missing values,?detecting outliers, etc. “

When should you build a model ?

You start building immediately after feature selection, with independent features.

Why is Model Building necessary ?

Building a machine learning model helps businesses in predicting the future.

How to build a model?

Scikit-learn?is used to build machine learning models.

Basic Steps to create a machine learning model:

1. Create two variables to store Dependent and Independent Features separately.
2. Split the variable(which stores independent features) into either?train,?validation,?test sets?or use?Cross validation?techniques to split the data.

Train?set - To train the algorithms

Validation?set - To optimize the model

Test?set - To evaluate the model.

Cross validation techniques?are used to split the data when you are?working with small datasets.

3. Build a model on a training set.

4.?What models can you build?

Machine Learning algorithms are broadly categorized into two types,?Supervised,?Unsupervised machine?learning algorithms.

Predictive models?are built using?Supervised Machine Learning Algorithms.

• The?models?built using?supervised?machine learning algorithms are known as?Supervised Machine Learning Models.
• There are?two?types of?Supervised Machine Learning Models?that can be build:
• —?Regression models: Some of the regression models are Linear Regression, Decision Tree Regressor, Random Forest Regressor, Support Vector Regression.
• —?Classification models: Some of the classification models are Logistic Regression, K-Nearest Neighbors, Decision Tree Classifier, Support Vector Machine(classifier), Random Forest Classifier, XGBoost.
• Unsupervised?machine learning algorithms are?not used?to build?models, rather they are?used in either identifying hidden groups/clusters in the data or to reduce dimensions of the data.
• Some of the unsupervised learning algorithms are?Clustering?Algorithms(like K-means clustering, etc),?Dimensionality Reduction Techniques(like PCA etc).

Step 8 — Model Evaluation

Source: Pixabay

What’s Model Evaluation?

In simple model evaluation means checking how accurate the model’s predictions are, that is determining how well the model is behaving on train and test dataset.

When to evaluate the model ?

As soon as model building is done, the next step is to evaluate it.

Why is model evaluation necessary?

In general, we will build?many?machine learning models by using different machine learning algorithms, hence evaluating the model?helps in choosing a model which is giving best results.

How to evaluate a model?

We use the?Scikit-learn?library to evaluate models using evaluation metrics.

Metrics are divided into two categories as shown:

1. Regression Model Metrics: Mean Squared Error, Root Mean Squared Error, Mean Absolute Error
2. Classification Model Metrics: Accuracy (Confusion Matrix), Recall, Precision, F1-Score, Specificity, ROC (Receiver Operator Characteristics), AUC (Area Under Curve).

Step 9: Model Optimization

Source: Pixabay

What’s Model Optimization?

Most of the machine learning models have some hyperparameters which can be tuned or adjusted. For example: Ridge Regression has hyperparameters like?regularization term, similarly Decision Tree model has hyperparameters like?desired depth?or?number of leaves?in a tree.

The process of?tuning these hyperparameters?to?determine the best combination of hyperparameters?to increase model’s performance?is known as?hyperparameter optimization?or?hyperparameter tuning.

When to optimize the model?

After calculating the Evaluation Metrics, we will choose the models with the best results and then?tune hyperparameters to enhance the results.

Why is Model Optimization necessary?

Optimization increases the performance of the machine learning models which in turn increases the accuracy of the models and gives best predictions.

How to do Model Optimization?

We make use of libraries like?Scikit-learn?etc or we can use frameworks like?Optuna?to optimize by tuning hyperparameters.

Hyperparameter tuning approaches include:

• Grid Search
• Random Search
• Bayesian Optimization
• Genetic Algorithms

Step 10 - Conclusion

Source: Pixabay

Finally, we will choose our?hyperparameter optimized model with the best metrics?and use that model for?production.

After all these steps, if you are still not happy with the machine learning model’s performance, then you can?repeat?the entire process starting from?Step 2?through?Step 9. Remember, Machine Learning is an?iterative,?hit and trial process?and its performance also depends on the sample of the data we gathered.

That’s it for this blog. I tried my best to keep it as simple as possible, and I hope you all got an idea on how to build and optimize a machine learning model.

As part of this series, we will implement all the above mentioned steps on?Telco Customer?data and come up with the best model to predict whether a customer churns.

Thanks for reading!!

This guest blog was written by Jaanvi .