Exclusive: Interview with Jeremy Howard on Deep Learning, Kaggle, Data Science, and more

Jeremy Howard, @jeremyphoward, is a true rock star in the world of Data Science. He was a precocious child, receiving some of the highest scores on tests in Australia, but was bored in school. He began an entrepreneurial career at age 12, selling pirated computer games, and was hired by McKinsey at age 18, as a self-taught computer and data analysis wizard. After a few years there he started Optimal Decisions Group which used data analysis to help insurance firms increase profits. His second starup, FastMail, was a very popular email provider. After successfully selling both in late 2000, he briefly retired, and took up hobbies like learning Chinese and building amplifiers.

Looking for an intellectual challenge, he entered a competition in 2010 at Kaggle, and was surprised to win the first place. He joined Kaggle as President and Chief Scientist and helped grow Kaggle to its dominant position. He left Kaggle in Dec 2013. In 2014 he started Enlitic with the mission of using Deep Learning to improve medical diagnostics and clinical decisions.

I first met Jeremy at KDD-2011 (?) conference, where he gave an unforgettable talk about Deep Learning. He did not have any slides or projector, but took a marker and proceeded to write on a white board (the only such speaker I can remember in history of KDD), explaining his ideas with surprising clarity and brilliance.

Jeremy's latest startup is fast.ai - read the details below.

See also an in-depth profile of Jeremy and Enlitic in Sydney Morning Herald (May 2016), and his TED talk: The Wonderful and terrifying implications of computers that can learn which gathered almost 2 million views.

Gregory Piatetsky, Q1. Tell us about your latest start-up fast.ai - what is it planning to do? How is your course "Deep Learning for Coders" different from other Deep Learning courses?

Jeremy Howard: There are a number of deep learning courses available online, but none of them met what we felt were the most important needs. We wanted to show people how to select and use the most effective deep learning techniques for their problems. And we wanted to make it as accessible as possible, without dumbing it down.

Previous approaches were either highly mathematical (such as the Oxford course) or too high level to be of much use in solving anything but the most basic problems (such as the Udacity course).

We have seen again and again that deep learning can provide state-of-the-art results, but to get these results requires getting a lot of little details right. And these little details are not shared in papers or in books or in online courses. There the kinds of things which get discussed directly among practitioners. Furthermore, we have seen very little discussion about important practical matters like: how to train your model in a reasonable amount of time, spending a reasonable amount of money.

We realised based on our analysis of and to and solutions to a number of deep learning projects, that the most important thing for us to teach is transfer learning. That refers to using existing models, which have already been trained on large datasets, to provide a helpful starting point for your model. Using transfer learning can speed up training time by many orders of magnitude, provide much more accurate models, and require much less data.

We also wanted to separate out the latest research fads from those things which really work. So we made sure that we only taught methods which we could actually show can provide state-of-the-art results on real-world problems. We've heard from a lot of people now that our deep learning MOOC has allowed them to dramatically improve the accuracy and speed of their model, so it seems to be working!

GP: Q2. Before fast.ai, in 2014 you founded Enlitic, whose goal is to use deep learning to make doctors faster and more accurate, initially in radiology. What progress did they make and how do they compare with trained radiologists?

JH: I don't know the latest, because I haven't been there for quite a few months. However everything I saw in my time studying medical deep learning showed that the opportunities here are enormous. It is a huge field, with many specialties and subspecialties, and everywhere we looked we saw opportunity. Most importantly, the opportunities have the potential to save lives, and greatly reduce healthcare costs, particularly in the developing world; this is where the needs are greatest.

(GP: Sydney Morning Herald reports:Enlitic pitted its algorithm against four top radiologists. The humans failed to spot 7% of the cancers. Enlitic identified them all. The humans incorrectly diagnosed cancer in 66% cases, Enlitic in 47%.

GP: Q3. What are the obstacles for adopting Enlitic and similar automated technology in healthcare?

JH: One of the greatest obstacles is the lack of integrated datasets - that is, datasets that show a history of medical tests, interventions, and outcomes, over a long period of time, links together for each patient. It is only with such a dataset that you can build models which can provide diagnoses and treatment recommendations based on actual medical outcomes, rather than initial diagnostic guesses.

Another obstacle is the lack of data scientists working in this field. I am surprised at how many smart and capable people are deciding to spend their time on relatively low impact areas like advertising technology, product recommendations, and minor social network features. Also, a lot of deep learning researchers are focused on "building a brain", rather than on solving current problems of most significance to humanity.

A particular obstacle that surprised me is that medical experts are so specialised, that it is hard to find anybody who can provide educated advice about more general medical problem solving. Deep learning has the ability to solve problems right across the medical spectrum, so the traditional specialised approach to medicine quite a roadblock.

GP: Q4. You are probably most well-known for being a Kaggle top-ranked competitor and later Kaggle President. What were the highlights of your time at Kaggle? What advice do you have for competitors that want to improve their Kaggle Ranking?

JH: My time competing on Kaggle was the highlight for me - in fact, I learned more about machine learning during that time than in the two decades prior to it. Another highlight has been the great enjoyment I have taken in the last few months studying a number of Kaggle datasets in some depth, in preparation for our course; it's been fascinating to see how some recent advances in deep learning makes it possible to get (what would have been) the highest rank in some competitions very quickly and easily.

For competitors who wish to improve their ranking, or indeed for any machine learning practitioners who wish improve their skills, my advice is simple:

submit an entry to a competition every single day.

Ideally, try to spend at least 30 minutes creating that entry; but even just spending five minutes tweaking some parameters is better than nothing. If you submit thing every day, then by the end of the competition will have learned a great deal, and you will learn even more when the winners blog posts come out. In your day-to-day work you will have very few, if any, opportunities to work on such rigorously defined datasets and metrics, and you certainly will not have a chance to benchmark yourself against some of the world's best data scientists.

Read the rest of the interview on KDnuggets,

Exclusive: Interview with Jeremy Howard on Deep Learning, Kaggle, Data Science, and more

https://www.kdnuggets.com/2017/01/exclusive-interview-jeremy-howard-deep-learning-kaggle-data-science.html