Two Poles Apart: An Explainer of ML and DL

All of a sudden everyone is talking about machine learning and deep learning, whether or not they understand the difference between them. People from non - Data Science background would also have heard of these terms. 

This article talks about what Machine Learning and Deep Learning is and how they are different from each other.

Let’s start off with the simple definitions of each :

MACHINE LEARNING : Most popular and recognized definition of Machine Learning was given by Tom Mitchell,

“A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E “

Suppose we want to use Machine Learning algorithms for predictions. For example, you want to predict the salary of a person based on his age, experience and all other factors. A simple linear line can help us make those predictions. Even if that line does a decent job in predicting, we might want to see or measure the performance. So for measuring or improving the performance, we could collect more data points (‘Experience’). We could also improvise on our model by adding in more variables and then creating different line predictions for them. 

Let’s move on and define Deep Learning.

DEEP LEARNING : Even though Deep Learning is not new since it has been there for a couple of years but nowadays the hype is increasing and therefore Deep Learning is getting all the attention. 

“Deep learning is an Artificial Intelligence (AI) function that imitates the workings of the human brain in processing data and creating patterns for use in decision making. Deep learning is a subset of machine learning in artificial intelligence that has networks capable of learning supervised from data that is unstructured or unlabeled.”

Suppose we want to predict whether the images are of a cat or a dog. If this problem was to be done by Machine Learning algorithms, we would have defined features like the animal has whiskers or not, if the animal has pointed ears or not and other facial features. So we would define all the facial features and let the system decide which feature is more useful in classifying whether it is a cat or a dog. 

Now what does deep learning do in this specific problem? It is one step ahead of Machine Learning. It automatically finds out the most suitable and apt features which are important for the classification. There is no need to manually define the features. 

Deep learning algorithms first define the edges that are the most relevant to find out a Cat or a Dog. It then builds this hierarchically to find which combinations of shapes and edges we can find. After consecutive hierarchical identification of complex concepts, it then decides which of these features are responsible for finding the answer. 

Moving forward to the second part of the article - Comparison of the two techniques : 

• Hardware Dependencies : Machine learning algorithms can work on low end machines but deep learning requires more powerful and high end machines. Deep learning requires a GPU for processing of the data since a lot of matrix multiplication operations are done. 
• Feature Engineering : Feature engineering is the process of using domain knowledge to extract features from raw data via data mining techniques. This is a very tedious process both in terms of time and expertise. As stated earlier, in Machine learning algorithms, these features have to be manually extracted depending on the domain knowledge of the problem and then the features are coded and fed to the system. Deep Learning algorithms on the other hand, try to learn the high level features on its own. That is why it is said that deep learning is one step ahead of machine learning. There might be some features that we didn’t extract while performing machine learning algorithms. In deep learning you do not have to worry about these things since they automatically learn different features and layers. 
• Data Dependencies : The major difference between the two is how well these techniques perform as the scale of data increases. Deep learning algorithms do not perform well when the data is small because they need large amounts of data to learn and identify different features. 
• Execution Time : Usually deep learning algorithms take a long time to train because there are so many parameters to train. Whereas, machine learning algorithms take less time to train as we feed them the features. This is by personal experience - in deep learning you take around 90% or more of data in training sets and the rest 10% in validation and testing. Whereas in machine learning, you only take 70-80% data in the training set and the rest in testing. Although in testing time, deep learning shows faster results as they take less time to run compared to machine learning. 
• Problem Solving approach : Suppose the problem statement comprises two or three tasks together. In machine learning, it is always advised to break down the problem statements and solve them individually and then combine the results in the end. Deep learning algorithms on the other hand can solve the problem at once end-to-end. 
• Interpretability : The performance that deep learning gives towards a problem statement is quite excellent and near to human performance. But it does not reveal why it came to the conclusion of that particular problem statement. Mathematically you can find out which nodes of a deep neural network were activated, but we don’t know what these neurons were supposed to model and what these layers of neurons were doing collectively. So we fail to interpret the results. It becomes like a black box model where the model does come to the conclusion but we can’t understand why it happens. Contrary to this, machine learning algorithms like decision trees give us crisp rules and boundaries as to why it chose what it chose so the interpretation of results becomes easier. This is why industries think twice before using deep learning algorithms. 

In short, the difference between the two can be given as :

Hope that this article was insightful. Thanks for reading.

Happy Learning! :)