The Deep Learning Roadmap
Carlos E. Perez
Author The Deep Learning Playbook, Artificial Intuition, Fluency & Empathy, A Pattern Language for Generative AI and Long Reasoning AI
It just occurred to me, that after a couple of years tracking Deep Learning developments, that nobody has even bothered to create a map of what’s going on! So I quickly decided to come up with a Deep Learning roadmap. A word of warning, this is just a partial map and doesn’t cover the latest developments. Many of the ideas I write on this blog isn’t even covered by this map. Anyway, here’s a start of this and hope people start coming out of their labs to further expand on it.
The “Unsupervised Learning” part is from a talk by Russ Salakhutdinov and the “Reinforcement Learning” part is from a talk by Pieter Abbeel.
There are a ton of other ideas that are coming out of the edges as well as the center of this diagram. Also, I did not show the connections between the 3 center concepts. For example, you can use CNNs for Value Iteration and GAN and VAEs use DL networks. It’s a wild world in the Deep Learning space and you just never know how all of this gets re-arranged.
I’ve got a higher level map that starts off with this:
that possibly can stitch everything together in one “grand unified theory”. This is how I think it will all play out:
Unsupervised learning is the the ‘dark matter’ where we need a lot more clarity. It’s my conjecture that meta-learning (with context) is the approach to this. There is some evidence that is developing, but I cannot know for sure. Modular Deep Learning is already in the cards. There is sufficient evidence that this works well. Market Driven Coordination is still early stages, but I believe that the only real way forward is to have diverse architectures working on the same problem and “markets” are a known decentralized way to coordinate actions.
There’s still a lot to be done though and we just in the early stages of Deep Learning evolution:
One additional key issue outside of unsupervised learning is the need to bridge the semantic gap between connectionist and symbolic architectures.
If you think there’s a demand for more clarity in Deep Learning, then support this kind of effort by buying the “Deep Learning Playbook”.
Thinknowlogy is the world's only naturally intelligent knowledge technology, based on Laws of Intelligence that are naturally found in the human language. Open souce software.
7 年Deep Learning evolution? I am sure deep-learning networks are intelligently designed, while intelligent design is not supported by the theory of evolution :-)