Decoding Deep Learning with Yoshua Bengio
What do?Yann LeCun , Yoshua Bengio, and Geoffrey Hinton have in common? Besides being called the 'Godfathers of deep learning,' – it's their unique?school of thought .?
While Bengio likes attention mechanisms, LeCun believes?self-supervised learning ?is the answer to achieving truly 'conscious' AI systems . Meanwhile, Hinton's most recent 'forward-forward propagation' seems to be the?new feat in deep learning. ?
However, the question is, how far are we in achieving AGI, and is it the end goal of AI? If you ask Bengio, he will tell you why he doesn't like the sound of AGI, or rather the terminology of it. "It's hard to say. But, if we understand the principles of intelligence that make us intelligent, we can probably go beyond human intelligence," said Bengio.
In an?exclusive interview with Analytics India Magazine, he said if we understood the physics of flight, we might have built aeroplanes inspired by birds, but we have not built bird-like machines. "We have built aeroplanes that can do things that birds cannot do. Birds can do things that aeroplanes cannot," he added, drawing us an interesting parenthesis.?
Bengio believes that the guiding principle should be what's best for humanity. "There are two things that motivate scientists: One is understanding what intelligence is, and the other is doing something useful," said Bengio, "hopefully, we will be applying these two principles when we get there as well."?
"I believe that as more data is better, bigger networks are better, but we are still missing some important ingredients to achieve the kind of intelligence humans have," said Bengio.?
Read:?Think Like Yoshua?
Not a Fan of LLMs?
Talking to AIM, Bengio said he is not excited about large language models (LLMs). He said that while it is impressive, it is not technically exciting as there is a lot of engineering involved and no new ideas. "As far as I am concerned, we do not know all the details. But, we have a pretty good understanding of what is going on there," said Bengio.
"One of the things that impressed me the most is the progress in Generative models," said Bengio. He said he was involved in coming up with generative adversarial networks (GAN) in 2014. Besides this, he also lauded 'diffusion models' and said they were previously working on similar things based on denoising. "Some of the ideas behind this date back to more than ten years ago," added Bengio, saying that these are some of the areas that interest him, and he is impressed by the progress they have made recently.?
"I am also quite excited by the progress that is happening in the area of probabilistic machine learning," said Bengio, saying that this is another area which has the potential to address some of the bigger challenges in deep learning. He also said that another area probably dates back to almost a decade ago – i.e. causality, which is a field much older than statistics, econometrics, and social sciences into?machine learning . He believes that this could be transformative.?
"One of the biggest limitations of current machine learning, including deep learning, is the ability to properly generalise to new settings like new distributions – what we call 'out of distribution generalisation,' and humans are very good at that," said Bengio. He said there are good reasons to think that humans are good at that because they have causal models of the world. "If you have a good causal model, you can generalise to new settings," he added.?
TOP STORIES OF THE WEEK
The Vedanta Semiconductor Plant Has Gone with the Wind
Not long ago, Vedanta was celebrated for setting up a semiconductor manufacturing unit in India, alongside Foxconn, promoting 'Atmanirbhar' India. But, now, that seems to have hit a roadblock as they are finding it difficult to raise funds from the Indian banks.?
Recent reports suggest that MeitY has asked the duo to provide a detailed plan to gather the required expertise to run a plant at a scale that the government envisions. In addition, they have been advised to get a third partner to demonstrate the plant's technical capabilities.?Read the full story here.?
领英推荐
What Makes Linux the Go-to System for AI/ML Development?
At the heart of the Linux OS is its kernel. Name the world's most-used operating system; it uses the?Linux ?kernel and has about 42 per cent of the global market presence. It contributes to the infrastructure in modern cloud computing, where the OS supports about 90 per cent of the public workload on the cloud. It is not an exaggeration to say that if it were not for?Linux , the cloud as we know it now would not exist.?Read the full story here.
ChatGPT's New Rival Has Arrived
OpenAI's conversational chatbot?ChatGPT ?might have taken the internet by storm, but it has also opened doors to competitors. The new arch-rival called Chatsonic – developed by Writesonic – has finally arrived, and it claims to be like?ChatGPT ?but with superpowers.?Read more here.
PEOPLE & TECHNOLOGY
Is DeepMind Deliberately Avoiding 'Shallow' Generative AI?
While the world of AI, including?DALL.E 2 ,?Stable Diffusion , and the most recent?ChatGPT , is gunning on?Generative AI ,?DeepMind ?is laying low on the creative side of things and focusing more on some 'real' and breakthrough innovations. This includes developing AI agents for games like?AlphaGo ,?AlphaStar ,?and DeepNash ; the revolutionary protein folding model,?AlphaFold ; the most recent 'matrix multiplication algorithms' discovery platform,?AlphaTensor ; and auto-code generator?AlphaCode .?
In an exclusive interview with AIM, DeepMind's head for research for AI for science and reliability,?Pushmeet Kolhi , shares the latest advancements in the world of generative AI and more.?Read the full story here.
India's Future Relies on Data Engineering, Not Data Models
AIM recently got in touch with IBM's country leader of data, AI and automation, Siddhesh Naik, to understand some of the challenges companies face when deploying AI models successfully. He said: 'India has no shortage of data' and emphasised the need for AI governance.?Read the full story here.
AIM VIDEOS
A Solution for India's Waste Problem?
In our latest episode of 'Tech for Good,' we introduce 'Carbon Masters, ' which aims to reduce carbon emissions at the ground level by converting wastes into biogas and fertilisers.
AIM SHOTS >>
Founder - European AI excellence center (CEEIA)
11 个月It won't happen until regulators and developers teach it only ethical things; it's like rowing a boat with a single oar. Artificial intelligence must be educated in both notions of good and evil to be balanced and, consequently, achieve AGI.
Software, Systems, Simulations and Society (My Opinions Merely Mine)
11 个月It is interesting to read the thoughts of the Godfathers, and understand that they also make profound errors in reasoning. "However, the question is, how far are we in achieving AGI, and is it the end goal of AI?" Two separate questions (a third implied) that are not really answered. 1) Implied "What is #AGI?" There is no consensus on this, so no reasonable answers can be discussed or provided without first defining the term. 2) Are we close? No. Because not only is the term not defined, but we are still at the age of narrow simulated intelligence. There has been some progress in the last 70 years, but little progress yet to understand, much less simulate "general intelligence". Cognitive Architectures have provided the most comprehensive framework for understanding human intelligence to date. 3) Is AGI the "end goal" of AI? No. "But, if we understand the principles of intelligence that make us intelligent, we can probably go beyond human intelligence," said Bengio. AI can and has surpassed human intelligence in multiple narrow domains. It will therefore likely continue to do so. But there is no reason to conclude that having machines simulate "General Intelligence" is even a desirable goal, much less an achievable one.