Brain-Inspired AI Agents: Setting Goals to Improve Complex Task Performance

In my previous post, “The Silent Disruptors: Gen AI Agents and Their Impact on Your Life, Business, and the World,” I highlighted the emergence of Gen AI agents, which are AI systems powered by large language models that can independently reason and execute tasks. These agents are set to transform various aspects of work and personal life, from customer service to travel planning. I highlight the potential impact of AI agents on work automation and the need for executives to understand this rapidly evolving technology.

I have been diving deeper into figuring out how we can make autonomous AI agents perform skilled tasks where the environment is always changing, akin to our real world. Our goal is to have a task-oriented autonomous AI agent function independently to accomplish specific tasks or achieve predefined objectives, adjusting priorities, learning from past experiences, and executing tasks without constant human oversight.

Long-term memory and continual learning are important for advancing AI agent's skills. Current AI models typically have fixed knowledge cutoffs and can’t learn or update their knowledge base through interactions. Developing AI that can retain information from past interactions and continuously learn would be a significant advancement.

In Figure 1, three agents interact, make decisions, and manage tasks. The agents perform an action based on user input or its previous outputs, observe the output, and repeat till the task is completed. The state of the environment changes between input and output and is represented by the state transition function. The state transition function defines how the environment changes its state in response to the agent’s actions. The environment, in turn, provides the context in which the state transition occurs. The agent’s actions, the environment’s state, and the state transition function together determine the outcome of the agent’s actions. In most ways, this is how the organism's brain functions as well, where it responds to actions and responses from the environment.

As we know, many of our breakthroughs in the field of AI have come from leading researchers and scientists studying how complex organisms work in the real world. Geoffrey Hinton, the Godfather of AI, revolutionized the field by making computers learn as people do. Deep learning has revolutionized the field of artificial intelligence (AI), and it was Geoffrey Hinton and his collaborators who introduced this groundbreaking concept. Now, with the dawn of Agentic AI, we can take many cues from the way Prof. Hinton approached his problems.

Noteworthy Research:

Of real noteworthy, there were couple of amazing scientific breakthroughs in recent times in understanding of the brain.

The first breakthrough was made by a team at Cambridge University published in the journal Science, led by Michael Winding and his team.

The researchers from the University of Cambridge have created the first comprehensive wiring map of the insect brain (Figure 2), specifically the fruit fly (Drosophila melanogaster). This map reveals the intricate connections between different brain regions, providing a detailed understanding of how the insect brain processes information.

“Some of the architectural features observed in the Drosophila larval brain, including multilayer shortcuts and prominent nested recurrent loops, are found in state-of-the-art artificial neural networks, where they can compensate for a lack of network depth and support arbitrary, task-dependent computations. Such features could therefore increase the brain’s computational capacity, overcoming physiological constraints on the number of neurons. Future analysis of similarities and differences between brains and artificial neural networks may help in understanding brain computational principles and perhaps inspire new machine learning architectures.”

Second, researchers have mapped a tiny sliver of the human brain on an unprecedented scale, vividly detailing each brain cell or neuron and the intricate networks they form with other cells. The groundbreaking brain map, which was constructed by Harvard and Google researchers, reveals roughly 57,000 neurons, 9 inches (230 millimeters) of blood vessels, and 150 million synapses, or the connection points between neurons. The human brain is a vastly complex organ with about 170 billion cells, including 86 billion neurons. The study used a combination of advanced imaging techniques and machine learning algorithms to create the map.

In Figure 3, a single neuron (white) receives signals that determine whether or not the neuron fires. This image shows all of the axons that can tell it to fire (green) and all of those that can tell it not to (blue).

The human brain, with its estimated 86 billion neurons and trillions of synaptic connections, remains the gold standard for cognitive processing and adaptability. Unlike traditional AI systems, which often rely on rigid algorithms and extensive training data, the brain excels at learning from limited examples, generalizing knowledge across domains, and rapidly adapting to new situations.

The researchers also developed computational tools to identify likely pathways of information flow and different types of circuit patterns in the insect’s brain. They found that some of the structural features are similar to state-of-the-art deep learning architecture, as per the Cambridge University article.

From Neuroscience to Advanced AI Model Development:

What we see from these research breakthroughs is the incredible, intricate architecture matrix-like where everything connects with everything else. And this can be a new metaphor for businesses organizing AI Agents. By incorporating principles from neuroscience into AI design, researchers can create systems that can:

1. Learn more efficiently with less data
2. Adapt dynamically to changing environments
3. Exhibit greater creativity and problem-solving skills
4. Perform complex tasks with human-like flexibility

A matrix structure allows for an efficient use of resources because such agent networks will include specialists from various departments/fields. This reduces overhead costs and the time needed to complete a task/project, as any agent who needs information or guidance can coordinate with another agent to accomplish their task.

In a hierarchical structure, every agent reports to only one supervisor agent, which makes the framework too rigid.

To do our most exciting and innovative work in agentic workflow, we’ve got to get those connections working much more effectively than in most rigid hierarchical systems. We need to bump up the scale of these connections by several notches. Agents have to function autonomously as they’re forming and reforming. This will require a lot of trial and error. We must have a central set of platform services and a distributed network of empowered agents that have autonomy and are aligned against a specific business goal, and they draw services from the central platform. These are some of what we are discovering about how the brain functions, even in the simplest organism, and these ideas can be translated to AI models.

The brain’s ability to integrate information from multiple senses can inform the development of AI systems that can process and integrate data from multiple sources, such as computer vision, natural language processing, and audio processing. We are already seeing major progress in multi-model LLMs.

The brain’s reinforcement learning mechanisms can inspire the development of more effective AI reinforcement learning algorithms, such as Q-learning and policy gradient methods. Reinforcement learning (RL) is a machine learning approach that teaches agents how to solve tasks by trial and error. Deep RL refers to the combination of RL with deep learning. OpenAI has put out some good documentation on their work in RL: https://spinningup.openai.com/en/latest/spinningup/rl_intro2.html

The brain’s ability to transfer knowledge from one task to another can inspire the development of more effective AI transfer learning methods, such as fine-tuning pre-trained models.

Another core area we need to develop is better explainability of the AI models. The brain’s ability to explain and justify its decisions can inspire the development of more transparent and explainable AI systems.

Summary:

The fruit fly brain mapping study’s findings can contribute to the development of brain-inspired AI systems that better mimic the insect brain’s capabilities. This could lead to more efficient and adaptive AI systems that can learn and adapt in complex environments.

The wiring map provides valuable insights into how the insect brain processes sensory information, motor control, and learning. This knowledge can be applied to improve the design of deep learning models, enabling them to better understand and process complex data.

We need to ensure that the overall common patterns, governance, and organizing structure are intricate and networked, allowing teams of agents across the organization to work together seamlessly and adapt to changing circumstances without descending into complete chaos.

As our understanding of the brain continues to evolve, so will our ability to create more sophisticated and capable autonomous agents. The synergy between neuroscience and AI research promises to unlock new frontiers in machine intelligence, potentially revolutionizing fields from healthcare and scientific discovery to enterprise businesses and space exploration.