Artificial Intelligence #62 Is most deep learning research really a waste of time?

I saw an interesting comment last week which said that??Most Research in Deep Learning is a Total Waste of Time

I would not pay much attention to it except when I saw that it was from Jeremy Howard - creator of fastai ?who is indeed well respected in the industry

Let's first see what he says and then I will share my comments

I partly agree with the viewpoint - but also believe that the view is limited and that practitioners in general often miss this point about research by creating a false dichotomy.

Let's first see the exact comment (sources below) - emphasis mine in bold

1)?most of the research in the deep mining world is a total waste of time?

2)?it's a?problem in science?in general?

3) scientists need to be published which means they need to work on things that their peers are extremely familiar

4) there's nothing to encourage them?to work on things that are practically useful

5) so you get just a whole lot of research which is?minor advances?and stuff that's been very highly studied

6) and has no significant?practical impact

He gives two examples of domains he feels are neglected: transfer learning and active learning

My initial thoughts

a)?There are certain broader issues - ex funding, tenure etc in science that also play a part - but we are not referring to these

b)?Secondly, far from being a waste of time, almost every other week, we see amazing developments in deep learning research (such as GATO recently)

but now let's come to the real argument

I have seen many practitioners talk of teaching practical things?and create a false dichotomy between practise vs research

However, when it comes to practice and research

Both have their respective trajectories

Both have their respective timelines?

There is often a false dichotomy (A vs B) portrayed?between?the two

More specifically, here are some of my comments

a) Let's?not forget that the AI winter was not a winter for Geoffrey Hinton - who toiled away (aka engaged in research) for almost two decades to create backpropagation and gradient descent - ie a way to train neural networks

b)?Research, by definition, solves harder problems - that take longer - and need peer review. This does not have a practical impact initially (but later does)

c)?AI research - and research in general - is a differentiator. The future competitiveness of countries will be decided by their research prowess. Hence, a relatively smaller country like the UK - which has established research institutes - ranks much more highly?than?larger, well funded countries.

Definitely, research is not a waste of time. Teaching people only practical stuff sounds very practical but long term its not. They are also the same ones who will tell you that you don't need to know maths to understand AI (which is the same argument rephrased in another way)

d)?For pure practitioners, I strongly recommend reading Thomas Kuhn's work?on the structure of scientific revolutions about how science advances through incremental innovations and paradigm shifts. I cannot explain this any better than Kuhn. Highly recommended?

e)?An emphasis on research is also what differentiates MIT, Oxford, Cambridge, IITs, UCL, Tsinghua, Technion from others. These institutions will always play a leading role in AI and will be in high demand. A role which policy makers have recognised.?

Finally, the role of research and science is key because fundamental breakthroughs change competitive advantages?

This was highlighted very well by comments from the?2021 Nobel prize winner in Physics Giorgio Parisi??who pointed out the risks of the relative lack of funding for fundamental research in his native Italy in contrast to countries like South Korea.

Thanks to Dr?Saeed Al Dhaheri for the original share

source?Most Research in Deep Learning is a Total Waste of Time - Jeremy Howard | AI Podcast Clips?

Image source Geoffrey Hinton who epitomises to me the reason why we indeed need research in AI although it may not seem to yield immediate results