AI through 2030

Where we are going (prediction)

As I wrote last week in "The past informs the present" I think we are at the beginning of a decade long period, roughly 2020-2030, of innovation based on the transformer / diffusion approach. To recap:

40?years (1960-2000) primarily focused on expert systems

20?years (2000-2020) machine learning and deep learning

10?years (2020-2030?) transformer, diffusion, large language models...

The innovation happening over this decade will, if Ray Kurzweil is correct, lead to an AI system that can pass a "broad Turing test" by 2029. To be clear (in case you don't have time to read Ray's thoughts) he is NOT arguing that such a system will be sentient, conscious or self-aware as we use these terms to describe human beings. But rather, that a system will be developed that can convince a knowledgable interviewer of its humanness -- that is, be at least a convincing simulacrum of a human being.

There are several research avenues that will continue to progress the capabilities of today's multi-modal AI models, like GPT-4. First there are the enhancements already being added using techniques that are well understood: enlarging the dataset, increasing the number of parameters, improving the training process... all of which provided the enhanced functionality of GPT-4 over the earlier GPT-3 and 3.5 models.

Second will be upcoming changes in LLMs allowing them to complete an action -- OpenAI's "plugin" model (in limited availability test mode now) is one example. Also, challenges in managing "memory" of an ongoing interaction will be addressed (such as in this research -- Longformer). This will improve how the model acts as an agent, especially in circumstances where there is a long running goal in to achieve.

Another example of the power of the plugin model is to remedy current deficiencies in GPT-4 such as the ability to properly handle numerical facts and computation. Wolfram Alpha solves that particular problem with a plugin.

A third important advance will be in complex image processing in which many objects within a field of view must be independently identified. Meta's SAM (segment anything model) is an example of research in this area.

This set of advances, along with a few that we don't know about yet, will not result (directly) in the development of "artificial general intelligence" (AGI) but will provide an increasingly reliable agent which responds in a humanlike way to our inquiries and can complete a wide range of tasks on our behalf. Expect to have "co-workers" who are able to work at your direction, substantially expanding your productivity and releasing you from menial tasks to focus instead on creativity, critical thinking, and interpersonal connections.

But there is a missing spark - what we might call "agency." It is unusual for Aristotle and Jerry Maguire to be quoted in the same sentence but: Where is the ? ο? κινο?μενον κινε? (unmoved mover), the first cause, or as Jerry Maguire says in the movie of the same name:

[screaming] Show me the money! Show me the money!

Or, show me the motivation!

Because as good as this 2020-2030 artificial intelligence could be as an assistant, it is still just taking direction from a human being. In the next phase, post-2030, we may see a change in how these machines develop motivations and thus act on their own. Some argue that this could be an "emergent" behavior, others that some new innovation is required. I am in the latter camp -- I believe human motivation emerges from our physical state and the pains and pleasures that our nervous system experiences. These are the drivers that at a very base level make us avoid cold and seek food. Our "first mover" or motivation is to improve our perception of our experience - a complicated interconnection between limbic and autonomic systems help humans regulate our responses to external and internal stimuli and through that regulation, allow us to have emotions, behavior, motivation, and memory.

So does the next evolution of machine learning take us to that next step closer to actual human intelligence? Do we need a machine version of all human sensory systems, such as sight, sound, touch, taste, and smell? Sight and sound are already here. Are motor systems required? Does true AGI only come once the AI is embodied in a robotic form? Is there an analogue to the human endocrine system? Which is to say, the release of hormones from the pituitary gland that helps to control various bodily functions, including stress response, hunger, thirst, and sexual behavior?

And as we evolve memory systems: how do all of these systems (sensory, motor, endocrine) contribute to the formation of new memories and the consolidation of long-term memories?

For now I will leave it with the prediction: after spending a decade (2020-2030) evolving the current versions of LLMs, we will be embarking upon a new phase of innovation that will explore how machines experience the world and how that experience informs and motivates them. To my mind this will be the most critical stage in developing alignment in our new machine friends.