Machine Learning Techniques at a Glance

A lot of knowledge is shared about Machine Learning and how it is changing the future. Following article tries to give an overview of all the techniques and models which can be used to solve a business problem using ML. This can be used as step guide to identify specific model which will be best suited to solve the problem.

First there must be a problem which needs to be addressed for one to proceed with ML. Just implementing any model without any vision further can be of little use since the model cannot be fine-tuned according to the problem one wishes to solve.

Snapshot of Machine Learning Techniques

Once the problem has been identified, it can be categorized as case of supervised or unsupervised learning.

       I.           Supervised Learning

In this method the data is labelled as input and output. The aim is to identify hidden relationship/patterns between the two.

     II.           Unsupervised Learning

Here the data is not labelled. The complete task of labeling the data and then identifying the patterns lies with machine.

Now, if the technique required is as supervised we further identify the problem as a classification or regression.

1.      Classification

A.     Lazy Learning using K-Nearest Neighbors

What?  This model uses information from the K-nearest neighbors to classify.

When to Use? The relationships between the features and the target are numerous, complex, and difficult to comprehend. In other words, if the target is difficult to define but you know where the variable will fit once you see it.

Conversely, if there is a lot of noise in the data and no clear distinction among the groups, K-NN might not be able to find the appropriate boundaries.

Tuning? The model can be tuned using an appropriate “K”.

Use Case: Classifying a food type into vegetables, fruits, proteins based on features like, sweetness, crunchiness

B.     Probabilistic Learning using Na?ve Bayes

What? This model uses probabilities for classifying each outcome based on the observed features.

When to Use? If the outcome has a requirement of considering all the features even if their individual impact is low but collectively they might have a bigger impact.

Tuning? Laplace Estimator: This replaces zero probability of a feature. The outcome is calculated using a multiplicative method. If probability of any feature is zero, while other features are still dominant, then the outcome is forced to zero. Laplace estimator replaces this zero-probability feature to eliminate this anomaly.

Use Case: Email classification, such as a spam

C.     Divide and Conquer using Decision Trees

What? Uses a flowchart like tree structure to model relationships between features and outcomes.

When to Use? Widely used technique to model any type of data. Not an ideal fit if data has a lot of nominal features resulting in over-fitting and a complex tree.

Tuning? Pruning is used to limit the number of branches a tree should have. This helps to reduce over-fitting of the data.

Use Case: Eligibility models for credit disbursement.

2.     Regression

A.     Multiple Linear Regression

What? Defines a relationship between a single dependent variable and one or more independent variables

When? Modelling complex relationships among data elements, estimating the impact of a variable on the outcome and extrapolating into the future. These are also used for hypothesis testing.

Tuning? Adding non-linear relationships, variables transformations like converting a numeric variable into a binary variable.

Use Case: Forecasting models by identifying key drivers.

B.     Decision Trees

What? Prediction is made by averaging the values of examples that reach the leaf.

When? If there is a need to identify key features from a large sample of features. Also, if the impact of features i not critical then this method comes in handy.

Tuning? Building model trees which replaces the leaf nodes with regression models. 

Use Cases: Estimating the quality of wines based on given factors

The below mentioned models are for the problems classified as unsupervised.

3.     Association – Finding Patterns:

Unsupervised method to identify patterns in a large database {a, b, c -> d}

A.     Apriori Model

What? This algorithm identifies association rules using two measures: support and confidence. Support measures the frequency of occurrence of an item set while confidence measures the predictive power of the rule.

When? When the transaction database is large with several features and eye balling is not feasible.

Tuning? Pre-defining the cut-offs for the confidence and support can help eliminate inexplicable rules.

Use Cases: Market Basket Analysis which can be used further for promotions and assortment.

4.     Clustering:

Unsupervised learning to identify clusters/groups of similar items

A.     K-Means Clustering

What? This algorithm assigns each of the examples to one of the predefined K clusters. The goal is to minimize the differences within each cluster and maximize the differences between the clusters.

When? The number of clusters to be formed is known in advance.

Tuning? Setting the right value for K will help improve homogeneity of clusters, whereas there is also a risk of over fitting of the data.   

Use Cases: Market Segmentation - If 3 distinct marketing campaigns are sanctioned, then the market segmentation can be done by identifying 3 clusters and targeting each campaign to the respective clusters.