Data is all you need

Generally speaking, Deep Learning is known for its insatiable appetite for labeled training data. It is getting much easier now to build, train and test a deep neural network for almost any task including computer vision, speech recognition, sentiment analysis and machine translation. However, the main challenge is how to get more labeled data.

Deep Learning is known for its insatiable appetite for labeled training data

Having experts in vision, speech or text domains to directly label a large amount of data is actually an expensive process in terms of time and cost. Here are some important approaches that try to avoid asking for additional labels.

Active learning: The main idea is to label only data points which are estimated to be most valuable to the model. This could save a lot of wasted time consumed in labeling useless data points, instead, focus only on the interesting data points that could add value. For example, choose to manually label the new data points that could be confusing to the current model, or near to the decision boundary of the classifier. Accordingly, do not waste time labeling data points that are very clear to the current model, or far from the decision boundary. Different strategies for determining which data points should be labeled are described briefly here.

Semi-supervised learning: The main idea is to use not only the small labeled training set, but also exploit a larger, easy to collect unlabeled data set. Accordingly, semi-supervised learning falls between unsupervised learning and supervised learning. It is found that large unlabeled data, when used with small labeled data, can lead to considerable improvement over the pure unsupervised learning. Recently, Generative Adversarial Networks, GANs, have been used to regularize decision boundaries.

In Transfer learning, we would like to leverage the knowledge learned by a source task to help learning another target task. For example, a well-trained, rich image classification network could be leveraged for another image target related task. In this way, we actually do need much labeled data for the target task. A related work is Multitask learning. Basically, we want to exploit the related tasks where learning a tasks should enhance the other. In other words, improving performance of a number of tasks simultaneously by optimizing all network parameters using samples from these tasks. For example, we would like to have one network that can classify an input face image as male or female, and at the same time can predict its age. Here we have two related tasks one is a binary classification task and the other is a regression task. It is clear that both tasks are related, and learning one should enhance learning the other.

Transfer Learning and Multitask learning are two vital approaches for Deep Learning

Regards