What is deep learning? Why is this a growing trend in machine learning?

The “deep” in “deep learning” refers to the number of layers through which the data is transformed. Deep learning has a large number of layers as compared to classical neural networks. More layers capture more statistical invariances. Moreover, deep Boltzmann machines are universal approximators. In deep learning, each level learns to transform its input data into a slightly more abstract and composite representation.

Deep learning neural networks are usually trained by using iterative, gradient-based optimizer that merely drive the cost function to a very low value, rather than the linear equation solvers used to train linear regression models or the convex optimization algorithms with global convergence guarantees used to train logistic regression or SVMs.

The classical approach to training neural networks is to minimize a (regularized) loss using backpropagation, a gradient descent method specialized to neural networks. Modern versions of backpropagation rely on stochastic gradient descent (SGD) to efficiently approximate the gradient for massive datasets.

Recently, Restricted Boltzmann Machines (RBMs) are used for deep neural networks that belong to so called Energy Based Models. RBMs achieved state of the art performance in collaborative filtering. In deep learning architectures the idea of energy is used as a metric for measurement of the models quality.

Five main reasons why deep learning is so popular

1. The deep learning networks can be efficiently implemented on massively parallel graphics processing units (GPUs).
2. They are easy to implement.
3. Deep learning networks can handle huge amounts of data
4. Deep learning networks can perform feature extraction and classification in one model.
5. As more and more data and computation power becomes available the use of deep learning will increase.