Not more, get better data!

There is often this tendency to just dump in whatever you have and let the Deep Learning model figure it out. Indeed, this is one of the biggest promises of?#deeplearning?- minimal feature engineering. But then again, those heavy Deep Learning models come with their own requirements on data, and a common solution has been to amass and throw more data at the model. I believe that such an approach is the lazy way out and it only spoiling us and dumbing us down collectively.

Whatever happened to the importance of parsimony? There are generations of Data Science professionals who have painstakingly learnt the importance of being frugal with data. The importance of parsimony - doing more with less. I myself have built several predictive models with less than 100 examples. To do so, I needed to -

• be very involved in feature engineering,
• really question the importance of each predictor,
• carefully evaluate the model for stability and generalizability
• be creative in sampling and use statistical methods for estimation

I wasn't alone in working with low volumes of data, it was the case across the board. Many 'older' data science professionals like myself ascribe value to the above steps. But it seems that these steps are losing their importance. More and more practitioners I speak to aren't concerned, or simply not aware of these considerations. Machine Learning seemingly has become a largely automated task, reduced to "just getting the best accuracy" from the predictive model.

This popular XKCD comic doesn't seem funny to me anymore. It's just scary.

"More data beats clever algorithms, but better data beats more data."

Coming from?Peter Norvig, you can be sure this is wisdom from decades of experiments, experiences from the 'front line' of 'cutting edge'. It is not just him. Several other thought leaders with decades of experience under their belt have the same opinion - good data is far more important than more data.

Not all situations requiring predictive modeling have high volumes of data. Not always do you need the maximum possible accuracy from the model. Not always do you need all possible homeomorphisms of the data to automatically generate powerful features. Not always do you really need that data hungry algorithm promising the highest accuracy!

Remember that in the industry, you employ Machine Learning to solve a problem. Employing a certain algorithm is rarely the goal and the end. Many a time, you do as much as you need to do to solve the problem and move on to the next problem waiting for you. Often there are constraints (business constraints, infrastructure constraints, data volume and many more) that severely limit the kind of modeling algorithms you can employ. In such situations, you won't the easy way out - dumping everything into a Deep Learning model.

Focus on the data.

Use your domain understanding to perform proper cleanup.

Derive features that inject your understanding into the process.

Indeed, a well crafted dataset with the right features can not only get you great results with classical?#machinelearning?algorithms, but can also help you get the most out of your state of the art #deeplearning / #artificialintelligence model. It can never harm you.

Don't lose the 'art' in Data Science! You can do much more than relying on a chain of matrix multiplications to figure everything out. Add your invaluable knowledge, your wisdom, your domain expertise to the process and make it truly awesome.