Emerging Alternative Artificial Intelligence Foundation Model Architectures Inspired by Brain Regions

Emerging Alternative Artificial Intelligence Foundation Model Architectures Inspired by Brain Regions

● Introduction:

The Convergence of Neuroscience and AI in Foundation Model Architectures:

○ The Rise and Limitations of Current Foundation Models:

The field of artificial intelligence has witnessed a significant paradigm shift with the emergence of foundation models.

These models, primarily based on the Transformer architecture, have demonstrated remarkable capabilities across a wide array of tasks, including natural language processing, computer vision, and multimodal applications (See Works Cited 1).

Leading examples such as OpenAI's GPT series, Google's BERT, and multimodal models like DALL·E have become cornerstones in the AI landscape (2).

These models are typically pre-trained on vast and diverse datasets, enabling them to learn general-purpose representations that can be adapted to specific downstream tasks through fine-tuning or prompting (2).

This pre-training on massive datasets allows foundation models to exhibit multitask capabilities and generalize to unseen data more effectively than traditional task-specific AI models (2).

However, despite their successes, Transformer-based foundation models are not without limitations.

One significant challenge lies in their computational complexity and substantial memory requirements, which can make training and deployment on resource-constrained devices difficult (6).

The self-attention mechanism, a core component of the Transformer, scales quadratically with the input sequence length, leading to inefficiencies when processing very long sequences (6).

Furthermore, the standard self-attention mechanism does not inherently capture causality, requiring modifications for autoregressive language modeling (6).

The interpretability of these large-scale models also remains a considerable hurdle, making it challenging to understand why they make certain predictions (6).

Recent theoretical work has also pointed out fundamental limitations, such as the difficulty in performing function composition when the domains are large (7).

These limitations suggest that while Transformer architectures have driven significant progress, there is a need to explore alternative approaches to further advance the field of AI foundation models.

○ The Promise of Brain-Inspired AI:

Brain-inspired artificial intelligence (BIAI) offers a compelling alternative by drawing inspiration from the structure, function, and information processing mechanisms of biological nervous systems, particularly the human brain (9).

The human brain, as a highly efficient and adaptable system, serves as a powerful model for creating more capable and sustainable AI.

BIAI aims to replicate the brain's ability to learn, adapt, and make decisions, often with the goal of achieving improved energy efficiency compared to traditional AI algorithms (9).

Potential advantages of BIAI include enhanced adaptability to new environments, better generalization from limited data, increased robustness to noise and perturbations, and improved pattern recognition capabilities (10).

Neuromorphic computing, a significant area within BIAI, focuses on designing both hardware and software that closely mimic the neural and synaptic structures of the brain to process information (9).

By emulating the brain's highly efficient and adaptable information processing, BIAI holds the potential to overcome some of the fundamental limitations of current AI foundation model architectures, paving the way for more sustainable and intelligent systems.

○ Focus on Brain Region Specific Inspirations:

The human brain exhibits a remarkable degree of functional specialization, with different cortical and subcortical regions dedicated to processing specific types of information and performing particular computational tasks (28).

For instance, the visual cortex is specialized for processing visual input, the auditory cortex for sound, the hippocampus for memory and navigation, and language areas like Broca's and Wernicke's areas for language processing.

This functional segregation suggests that drawing inspiration from the unique architectural and algorithmic principles of specific brain regions could lead to the development of AI foundation models that are similarly specialized for certain types of processing.

By mimicking these region-specific mechanisms, it might be possible to create AI models that are more efficient and effective within their intended domain of expertise.

Adopting a brain-region specific approach to designing AI foundation models could enable the creation of highly optimized architectures tailored for particular modalities or cognitive functions, potentially leading to significant performance gains and improved efficiency.

This modular approach, mirroring the brain's organization, could also facilitate the development of more interpretable and controllable AI systems, where the function of different components is more directly linked to known biological processes.

● Visual Cortex Inspired Foundation Models:

Architectures for Image Recognition and Processing

○ Structure and Function of the Visual Cortex:

The visual cortex, situated in the occipital lobe at the back of the brain, is the primary area responsible for interpreting and processing visual information received from the eyes (28).

This complex region is hierarchically organized into several distinct areas, labeled V1 through V5, each with specialized functions in visual perception (28).

The primary visual cortex, or V1, is the first cortical area to receive visual input and is characterized by its layered structure, containing various types of neurons, including simple and complex cells (28).

Simple cells in V1 are highly sensitive to specific features like the orientation of edges and lines, while complex cells respond to these features across a range of positions and orientations (28).

Beyond V1, other areas of the visual cortex, such as V2, V3, V4, and V5, process increasingly complex aspects of visual information, including color, motion, and object recognition (28).

Notably, visual processing in the cortex is often described as occurring along two main pathways:

The dorsal stream, which extends into the parietal lobe and is involved in processing spatial information and motion ("where");

And the ventral stream, which projects into the temporal lobe and is crucial for object recognition and form perception ("what") (34).

This hierarchical and parallel organization of the visual cortex, with its specialized cells and processing streams, provides a sophisticated model for how the brain analyzes and understands visual input.

○ Computational Models Inspired by the Visual Cortex:

The hierarchical and feature-based processing observed in the visual cortex has profoundly influenced the design of computational models for image recognition and processing.

Deep Neural Networks (DNNs), and particularly Convolutional Neural Networks (CNNs), are prime examples of AI architectures inspired by the visual cortex (36).

CNNs utilize convolutional layers to automatically learn hierarchical representations of visual features, starting from simple edges and textures in early layers to more complex shapes and objects in deeper layers, mirroring the processing stages in the visual cortex (36).

More recent advancements include models specifically designed to mimic the functions of particular regions within the visual system.

MovieNet, for instance, draws inspiration from the optic tectum, a visual processing center in the brainstem, to analyze videos in a manner similar to how the brain interprets moving images over time (39).

By processing video clips as sequences of brief visual cues, MovieNet has demonstrated remarkable accuracy in distinguishing between different dynamic scenes, even outperforming traditional AI models and human observers in certain tasks (27).

Another significant example is the object recognition system developed by Serre Lab, which employs a hierarchical architecture with computational units designed to emulate the behavior of simple and complex cells in the primary visual cortex (V1) (42).

This system uses Gabor filters, inspired by the receptive fields of V1 simple cells, and a max-like pooling operation, analogous to the pooling of simple cell responses by complex cells, to achieve robust object recognition across various image datasets.

○ Emerging Architectures and Research Directions:

Current research continues to explore novel ways to leverage the principles of visual cortex processing in AI foundation models.

One direction involves investigating the role of high-dimensional representations in DNN models of the visual cortex, with findings suggesting that models with higher representational dimensionality tend to exhibit better generalization performance and efficiency in learning new visual categories (44).

This suggests that the richness of the internal representations within these models is a key factor in their success.

Another active area of research is the integration of visual attention mechanisms into visual cortex-inspired models (38).

Visual attention, the brain's ability to selectively focus on relevant parts of the visual field, is crucial for efficient processing.

Computational models that incorporate attention mechanisms, often guided by the spatiotemporal features detected by visual cortex-like processing, have shown promise in tasks such as action recognition (45).

Future efforts are likely to focus on developing more dynamic and context-aware attention mechanisms that more closely resemble biological vision.

Furthermore, there is a growing interest in creating AI models that capture the temporal dynamics of visual processing in the brain, moving beyond the traditional focus on static image analysis (40).

Models like MovieNet, which process video as a sequence of events, represent a step in this direction.

Future research may explore how to better model the short-term memory and predictive capabilities of the visual cortex in handling temporal visual data, potentially leading to advancements in video understanding, autonomous driving, and other applications that require processing dynamic visual information.

● Auditory Cortex Inspired Foundation Models:

Architectures for Speech and Sound Processing:

○ Structure and Function of the Auditory Cortex:

The auditory cortex, located within the temporal lobe, is the primary processing center for auditory information in the brain (29).

It is organized into a hierarchical structure, including the primary auditory cortex (A1) and surrounding secondary and tertiary auditory areas (49).

A defining characteristic of A1 is its tonotopic map, a spatial arrangement where neurons respond preferentially to different sound frequencies, mirroring the organization of the cochlea in the inner ear (29).

The auditory cortex is particularly specialized for processing temporal sequences of sound, a crucial ability for understanding complex auditory signals such as speech and music (29).

Different regions within the auditory cortex appear to be responsible for analyzing various aspects of sound, including pitch, loudness, timbre, and the spatial location of sound sources (29).

For example, the anterior auditory cortex (AAC) is involved in processing complex sounds like speech and music, while the posterior auditory cortex (PAC) plays a role in sound localization (50).

The intricate organization and functional specialization of the auditory cortex highlight the brain's sophisticated mechanisms for analyzing and interpreting the auditory world.

○ Computational Models Inspired by the Auditory Cortex:

Computational models have increasingly drawn inspiration from the auditory cortex to improve AI's ability to process and understand sound.

Researchers have explored analogies between the auditory and visual cortices, suggesting shared computational principles adapted for different sensory inputs (52).

Deep neural networks (DNNs) have proven to be effective in modeling neural coding within the auditory cortex, particularly for speech recognition tasks (53).

Studies have shown that the internal representations learned by DNNs trained on speech exhibit a hierarchical structure that aligns with the processing hierarchy observed in the human auditory cortex, from acoustic features to phonetic, lexical, and semantic information (53).

Furthermore, brain-inspired architectures like cortico-thalamo-cortical neural networks (CTCNet) have been developed for specific auditory tasks, such as audio-visual speech separation (56).

These models draw inspiration from the neural circuits connecting the auditory cortex with the thalamus, aiming to leverage visual information to enhance the separation of speech from mixed audio signals, similar to how humans use visual cues to understand speech in noisy environments.

These computational models demonstrate the potential of mimicking the auditory cortex's processing strategies to advance AI in various audio-related domains.

○ Emerging Architectures and Research Directions:

Future research in auditory cortex-inspired foundation models is likely to focus on addressing current limitations in AI's ability to process sound in complex, real-world scenarios.

One direction involves developing models that exhibit adaptive gain control and noise filtering mechanisms similar to those found in the auditory cortex, which allows humans to maintain speech comprehension even in the presence of changing background noise (57).

Studies using DNNs as models of the auditory cortex have shown that these networks can indeed model the dynamic response patterns observed in the brain when adapting to sudden changes in noise (58).

Another promising area is the exploration of parallel hierarchical encoding of linguistic representations, inspired by how the human auditory cortex transforms acoustic speech signals into meaningful linguistic information (54).

By creating AI models with a processing hierarchy that mirrors the stages in the auditory cortex – from acoustic to phonetic, lexical, and semantic – researchers aim to develop more robust and context-aware speech processing systems.

Additionally, integrating multimodal information, such as combining auditory and visual cues for speech processing, as seen in models like CTCNet, is expected to be a key focus in future research, leading to AI systems with more human-like abilities to understand and interact with the world through sound (56).

● Hippocampus Inspired Foundation Models:

Architectures for Memory, Navigation, and Sequential Data Analysis:

○ Structure and Function of the Hippocampus:

The hippocampus, a seahorse-shaped structure located deep within the temporal lobe of the brain, plays a pivotal role in several critical cognitive functions, most notably the formation of new memories (30).

It is particularly essential for the creation of episodic memories, which are autobiographical records of events, and spatial memories, which involve information about locations and navigation (59).

The hippocampus is thought to be crucial for the initial encoding of these memories and their indexing for later retrieval (59).

Furthermore, the hippocampus is involved in processing sequential information, including the temporal order of events, which is vital for tasks such as planning and prediction (32).

It also interacts extensively with the neocortex, the brain's outer layer responsible for higher-level cognitive functions, to facilitate the consolidation of memories over time, transferring them from a more labile state in the hippocampus to a more permanent form in the neocortex (59).

This interplay between the hippocampus and neocortex is thought to be fundamental to how we learn and retain knowledge.

○ Computational Models Inspired by the Hippocampus:

The hippocampus's crucial roles in memory, spatial cognition, and sequence processing have inspired numerous computational models in artificial intelligence (62).

These models attempt to replicate various aspects of hippocampal function, such as the formation of associative memories, the encoding of spatial maps using place cells, and the processing of temporal sequences.

One notable concept is the memory-replay mechanism, inspired by the reactivation of neural activity patterns in the hippocampus during sleep or rest, which is believed to be important for memory consolidation (62).

AI models have incorporated this mechanism to enhance learning in reinforcement learning agents, allowing them to consolidate learned experiences and improve navigation strategies (62).

Interestingly, recent research has revealed a striking similarity between the memory processing in Transformer networks, a dominant architecture in AI, and the hippocampus (71).

This suggests that AI models might be employing mechanisms analogous to those found in the brain for memory consolidation, highlighting a potential convergence in how biological and artificial systems handle memory.

These computational efforts demonstrate the significant potential of drawing inspiration from the hippocampus to enhance memory, spatial reasoning, and learning capabilities in AI systems.

○ Emerging Architectures and Research Directions:

Future research in hippocampus-inspired foundation models is likely to explore several promising directions.

One is the development of dual-process learning systems that explicitly model the interaction between a fast-learning hippocampal-like component and a slow-learning neocortical-like component, mirroring the Complementary Learning Systems (CLS) theory of memory (64).

This could lead to AI systems that can rapidly acquire new information while gradually integrating it into a stable knowledge base, addressing the stability-plasticity dilemma.

Another direction involves incorporating more sophisticated memory consolidation mechanisms into AI models, including replay and offline learning processes inspired by hippocampal activity during sleep (64).

This could help improve long-term memory retention and prevent catastrophic forgetting in AI systems that learn continuously.

Creating models with enhanced pattern separation and pattern completion abilities, key functions of the hippocampus that allow for the encoding of distinct memories for similar experiences and the retrieval of complete memories from partial cues, is also an important area of focus (73).

Furthermore, researchers are exploring the use of hippocampal-inspired models for time series analysis and prediction, leveraging the hippocampus's role in processing temporal sequences (74).

These efforts aim to create AI systems with more robust and flexible memory and sequential data processing capabilities, potentially leading to advancements in areas like lifelong learning and autonomous navigation.

● Language Processing Inspired Foundation Models:

Architectures Drawing from Broca's and Wernicke's Areas:

○ Structure and Function of Broca's and Wernicke's Areas:

The processing of language in the human brain is a complex and distributed process involving several key areas, most notably Broca's area and Wernicke's area, typically located in the left hemisphere (31).

Broca's area, situated in the frontal lobe, plays a crucial role in language production, including the planning and execution of speech and the generation of grammatically correct sentences (31).

Damage to this area can result in Broca's aphasia, characterized by difficulties in producing fluent speech.

Wernicke's area, located in the temporal lobe, is primarily involved in language comprehension, enabling the understanding of spoken and written language and the interpretation of the meaning of words and sentences (31).

Damage to Wernicke's area can lead to Wernicke's aphasia, where individuals may produce fluent but often nonsensical speech and have difficulty understanding language.

These two areas are interconnected by a bundle of nerve fibers called the arcuate fasciculus, which facilitates communication between them, allowing for the integration of comprehension and production aspects of language (77).

○ Computational Models Inspired by Language Areas:

The functional specialization of Broca's and Wernicke's areas has long inspired computational models of language processing.

Early models, such as the Wernicke-Lichtheim-Geschwind model, proposed a modular view of language, where different brain regions are responsible for specific language functions, with information flowing between these modules (78).

While the neurological assumptions of this model have been refined over time, the concept of functional specialization remains influential.

Recent research has explored whether the emergent properties of large language models (LLMs) exhibit modularity that resembles the functional organization of the brain's language areas.

Studies investigating the effects of "lesioning" specific parts of LLMs have found some parallels with the language deficits observed in different types of aphasia, suggesting that certain components within these models might be specialized for semantic or syntactic processing, akin to Wernicke's and Broca's areas, respectively.

Computational models continue to be developed that aim to mimic the connectivity and processing within the brain's language network to perform tasks like language understanding and generation (81).

○ Emerging Architectures and Research Directions:

Future research in Natural Language Processing (NLP) may see a greater emphasis on developing foundation models with more explicitly modular architectures, drawing inspiration from the brain's language network.

This could involve creating models with distinct components specialized for different aspects of language processing, such as phonology, syntax, semantics, and pragmatics; and exploring how these components interact.

Researchers are also looking beyond just Broca's and Wernicke's areas to consider the roles of other brain regions involved in language, such as the angular gyrus for associating different types of language-related information and the prefrontal cortex for executive functions in language processing (76).

Another important direction is to create models that can better capture the nuances of human language, including the dynamic integration of semantic and syntactic information, the role of context in understanding meaning, and the ability to handle figurative language and common-sense reasoning (83).

Brain-inspired approaches that reflect the distributed and interactive nature of language processing in the brain might offer new pathways to achieve these goals, potentially leading to AI systems with a more profound and human-like understanding of language.

● Neocortex Inspired Foundation Models:

General Architectures for Sequential Learning and Pattern Recognition:

○ Structure and Function of the Neocortex:

The neocortex, the largest and most recently evolved part of the cerebral cortex, is the principal brain region responsible for higher-level cognitive functions, including sensory perception, motor commands, spatial reasoning, language, and conscious thought (52).

Its defining feature is a hierarchical, layered structure that is remarkably uniform across different functional areas, suggesting that a common set of computational principles might underlie its diverse capabilities (88).

A fundamental function of the neocortex is its ability to learn the underlying structure of the world from the continuous stream of temporal data received through our senses (48).

By identifying patterns and regularities in this sensory input, the neocortex builds internal models of the world that enable it to make predictions about future events, guiding our behavior and allowing us to interact effectively with our environment.

This capacity for sequence learning and prediction is considered a cornerstone of intelligence.

○ Hierarchical Temporal Memory (HTM):

Hierarchical Temporal Memory (HTM) is a biologically inspired machine learning theory developed by Jeff Hawkins and his team at Numenta, which aims to replicate the structural and algorithmic principles of the neocortex for learning and prediction (48).

HTM organizes its computation into a hierarchy of regions, each composed of columns of cells that learn and recognize patterns in the input data, represented as sparse binary distributed representations (SDRs) (74).

HTM is particularly well-suited for learning and predicting temporal sequences, exhibiting key properties such as online learning, the ability to make high-order and multiple simultaneous predictions, continual learning without retraining, and robustness to noise (48).

It has been successfully applied to a variety of tasks involving time series data, including anomaly detection, forecasting, and sequence modeling in diverse domains like finance, robotics, and cybersecurity (74).

○ Other Neocortex Inspired Architectures:

Beyond HTM, researchers continue to explore other ways to translate the computational principles of the neocortex into effective AI architectures for sequential learning and pattern recognition.

Neocortex-Inspired Locally Recurrent Neural Networks (NILRNN) are designed to mimic the local recurrent connections found in the neocortex's feedforward circuits, demonstrating strong performance in unsupervised feature learning from sequential data (92).

Dynamic predictive coding models propose a hierarchical framework for sequence learning and prediction inspired by the neocortex's ability to minimize prediction errors (98).

These models suggest that higher cortical levels modulate the temporal dynamics of lower levels, leading to the development of hierarchical representations of sequences across different timescales.

Additionally, some research focuses on creating models that integrate principles from different brain regions, such as combining hippocampal mechanisms for memory replay with neocortical learning to achieve autonomous knowledge consolidation during simulated sleep (64).

These diverse approaches highlight the ongoing effort to unlock the computational power of the neocortex for the next generation of AI systems.

● Graph-Based Foundation Models Inspired by Brain Networks

○ Brain Networks as Graphs:

The human brain, rather than functioning as a collection of isolated modules, operates as a complex network of interconnected regions (99).

These connections, both structural (anatomical pathways) and functional (correlated activity), form intricate patterns that are crucial for supporting cognitive processes.

This network organization can be naturally represented using the mathematical concept of graphs, where individual brain regions or neural populations are considered nodes, and the connections between them are represented as edges (99).

Graph Foundation Models (GFMs) are an emerging class of AI models that are pre-trained on extensive graph data and can be adapted for a wide range of downstream tasks involving graph-structured information (99).

This approach is particularly relevant for brain-inspired AI because it provides a natural way to model the non-Euclidean structure of brain connectivity, which traditional foundation models, especially those based on sequential or grid-like data, often struggle to represent effectively (99).

○ Approaches to Graph Foundation Models:

Research into Graph Foundation Models encompasses several distinct approaches, often categorized by their dependence on Graph Neural Networks (GNNs) and Large Language Models (LLMs) (99).

One primary approach involves enhancing existing graph learning paradigms through innovations in GNN architectures, pre-training techniques, and methods for adapting pre-trained models to specific downstream tasks (99).

This includes developing new types of GNN layers, exploring effective self-supervised pre-training objectives on large graph datasets, and devising strategies for fine-tuning or prompting GNNs for various graph-level, node-level, or edge-level prediction tasks.

Another direction explores the possibility of leveraging the power of LLMs for graph-related tasks by transforming graphs into a format that LLMs can process, such as sequences of tokens or natural language text (99).

Techniques like graph-to-token and graph-to-text transformation aim to bridge the gap between the structured nature of graphs and the sequence-based processing of LLMs.

Finally, a third category of research focuses on creating hybrid models that combine the strengths of both GNNs and LLMs (99).

This can involve using LLMs to extract features from node attributes or context that are then used by GNNs, or symmetrically aligning the embedding spaces of GNNs and LLMs to create a unified representation for graphs and text.

○ Potential and Challenges:

Graph Foundation Models (GFMs) hold significant potential for a variety of applications, including tasks directly relevant to understanding brain networks.

For example, GFMs could be used for node classification to identify different types of neurons or brain regions based on their connectivity patterns, for link prediction to infer potential connections between neurons or regions, and for knowledge graph reasoning to understand complex relationships within the brain's functional architecture.

Some early examples of GFMs, such as GraphAny for node classification and ULTRA for knowledge graph reasoning, have shown promising results in generalizing to new graphs and tasks.

Despite this potential, the field of GFMs also faces several challenges.

One major hurdle is the limited availability of large-scale, high-quality graph datasets across diverse domains, which is crucial for effective pre-training (100).

Evaluating the performance of GFMs, especially on open-ended tasks that might involve generation or reasoning, also presents difficulties due to the lack of standardized evaluation metrics and labeled data (100).

Furthermore, designing robust and expressive model architectures that can effectively capture the complex structure and semantics of graphs, and determining the optimal pre-training tasks and adaptation methods for different downstream applications, remain active areas of research (100).

● Neuromorphic Computing Architectures as Foundation Models

○ Principles of Neuromorphic Computing:

Neuromorphic computing is a rapidly evolving field that seeks to revolutionize computing by emulating the architecture and function of the human brain (9).

Unlike traditional von Neumann architectures that separate memory and processing units, neuromorphic systems aim to integrate these functions in a way that mirrors the brain's neural networks (9).

A core principle of neuromorphic computing is the use of spiking neural networks (SNNs), which communicate using brief, discrete signals called spikes, similar to biological neurons (9).

This event-driven approach allows for highly parallel and energy-efficient computation (9).

Other key principles include asynchrony, where neurons operate independently without a global clock, and plasticity, the ability of synaptic connections to change over time based on activity, enabling learning (9).

Neuromorphic systems often employ local learning rules, such as Spike-Timing-Dependent Plasticity (STDP), where the strength of a connection between two neurons is adjusted based on the precise timing of their spikes (9).

The goal of neuromorphic computing is to create more efficient, adaptable, and fault-tolerant computing systems that can excel in tasks where the brain demonstrates superior capabilities, such as pattern recognition, sensory processing, and real-time learning (9).

○ Spiking Neural Networks (SNNs):

? Sidebar: The Three Generations of Neural Networks:

First-Generation Neural Networks were based on neurons that were essentially threshold units.

A classic example is the "Perceptron."

These networks could perform simple linear classifications.

They lacked the ability to handle complex, non-linear problems, which led to limitations.

Essentially, they were using neurons that had a binary output, either 1 or 0, depending on if a threshold was reached.

Second-Generation Neural Networks introduced neurons with continuous activation functions, such as the sigmoid function.

This allowed them to handle non-linear problems using techniques like backpropagation.

This generation saw the rise of multilayer perceptrons and backpropagation algorithms, which significantly expanded the capabilities of neural networks.

These networks are the basis of much of what is considered "classical" neural networks.

Third-Generation Neural Networks are Spiking Neural Networks (SNNs).

They aim to more closely mimic the behavior of biological neurons by using spikes, or discrete events, for communication.

This event-driven nature offers potential advantages in energy efficiency and temporal processing.

They are a core component of neuromorphic computing.

In essence, the progression has been from simple threshold-based neurons to continuous-valued neurons, and finally to spike-based neurons that more closely resemble biological systems.

. . .

Spiking Neural Networks (SNNs) are considered the third generation of neural network models and are a cornerstone of neuromorphic computing (103).

Unlike traditional Artificial Neural Networks (ANNs) that process continuous values, SNNs operate using discrete spikes, making them more biologically realistic and potentially more energy-efficient (10).

In an SNN, neurons accumulate charge over time, and they only fire a spike when their membrane potential reaches a certain threshold (9).

This event-driven nature means that computation is sparse, occurring only when spikes are generated and transmitted, which can lead to significant power savings compared to the continuous activity in ANNs (10).

While SNNs offer many advantages, training them to achieve high accuracy on complex tasks has been a challenge due to the non-differentiable nature of spike events (106).

? Sidebar: The Challenges of Non-Differentiable Spikes:

The non-differentiable nature of spike events in Spiking Neural Networks (SNNs) presents a significant challenge for training because the most successful method for training deep neural networks, called backpropagation, relies on gradients.

Here's why:

Backpropagation and Gradients:

Backpropagation works by calculating how much each weight in the network contributes to the error in the network's output.

This calculation involves finding the derivative (or gradient) of the error with respect to each weight.

These gradients tell the algorithm how to adjust the weights to reduce the error and improve the network's accuracy.

Spikes are Discrete Events:

In SNNs, neurons communicate using brief, discrete signals called spikes.

These spikes are essentially on/off events, rather than continuous values like in traditional neural networks.

The Problem with Derivatives of Spikes:

The mathematical function that describes a spike is typically a step function (or something very similar).

Step functions are non-differentiable at the point where the spike occurs.

This means that at the crucial point of the spike, the gradient is either zero or undefined.

Hindrance to Weight Adjustment:

Because the gradient is often zero or undefined, the backpropagation algorithm doesn't get the information it needs to understand how to adjust the weights in the SNN to improve its performance.

The error signal cannot be effectively propagated backward through the network.

Search for Alternatives:

This lack of a clear gradient signal makes it difficult to directly apply backpropagation to train SNNs for complex tasks.

As a result, researchers are actively exploring alternative training methods specifically designed for the spiking nature of these networks.

These methods include techniques like converting pre-trained traditional neural networks to SNNs or developing new learning rules that work directly with spikes.

. . .

Current research efforts are focused on developing effective training methods, including direct training using spike-based learning rules and converting pre-trained ANNs to SNNs by mapping activation values to spike rates or timings (110).

The future of SNNs looks promising, with potential for applications in areas where energy efficiency and temporal processing are crucial.

○ Neuromorphic Hardware and Scalability:

To fully realize the potential of neuromorphic computing and SNNs, specialized hardware is essential.

Several neuromorphic chips have been developed, such as IBM's TrueNorth and Intel's Loihi, which are designed to efficiently implement spiking neural networks (9).

These chips often feature a massively parallel architecture, with many cores that can simulate neurons and synapses, and they integrate memory and processing to reduce energy consumption and latency (9).

Scaling neuromorphic systems to handle complex AI tasks requires addressing several challenges.

One key aspect is optimizing the sparsity of neural connections, inspired by the brain's own efficient wiring (11).

Another is the development of user-friendly programming languages and software tools that can facilitate the design, training, and deployment of large-scale neuromorphic networks (11).

Researchers are also exploring new materials and device technologies, such as memristors, to create neuromorphic chips with improved performance and energy efficiency (9).

? Sidebar: Memristors:

A memristor is an emerging type of nonvolatile electronic memory device that combines the functions of memory and a resistor.

It's characterized by its memristive property, meaning its resistance depends on the history of the voltage applied across it.

This unique characteristic allows memristors to mimic the behavior of synapses in the human brain, specifically synaptic plasticity, where the strength of a connection changes over time based on activity.

Key features of memristors include fast operational speed, low energy consumption, and small feature size, making them promising for applications in analog circuits for neuromorphic computing.

In neuromorphic architectures, memristors can be used to store synaptic connection strengths, enabling the co-location of memory and data processing in spiking neurons, which is a key principle of brain-inspired computing.

. . .

Initiatives like BrainScale aim to create scalable frameworks for online learning in SNNs, which is crucial for building large and adaptable brain-inspired AI models (107).

The progress in both neuromorphic hardware and software is steadily advancing, paving the way for the development of more powerful and practical brain-inspired foundation models.

● Scaling Brain-Inspired AI Foundation Models:

Challenges and Opportunities:

○ Challenges in Scaling:

Scaling brain-inspired AI foundation models to the level of current large-scale models like GPT-3 presents a formidable set of challenges (36).

The complexity of biological systems, particularly the human brain with its intricate network of billions of neurons and trillions of synapses, is a major hurdle in replication (21).

Our understanding of fundamental brain functions such as learning, memory, and consciousness remains incomplete, making it difficult to translate neuroscientific insights into robust and scalable AI algorithms (21).

Bridging the gap between the fundamental differences in structure and function between biological and artificial neural networks is another significant challenge (73).

While brain-inspired AI aims to mimic biological processes, the constraints of current computing hardware and the limitations of our knowledge often necessitate simplifications and abstractions.

Achieving the energy efficiency of the human brain, which operates on a remarkably low power budget compared to current AI systems, at scale is a persistent goal (73).

Additionally, many brain-inspired models, especially those designed for continuous learning, struggle with catastrophic forgetting, where learning new tasks causes the model to forget previously learned ones (73).

Training very large brain-inspired models can also be extremely resource-intensive, requiring vast amounts of high-quality data and significant computational power, which can limit accessibility and slow down research progress (4).

○ Opportunities in Scaling:

Despite the challenges, successfully scaling brain-inspired AI foundation models offers a wealth of transformative opportunities (27).

One of the most significant is the potential for creating AI systems that are far more energy-efficient and sustainable than current large-scale models, addressing the growing environmental concerns associated with the energy consumption of AI (11).

Brain-inspired architectures, particularly those based on neuromorphic computing and spiking neural networks, inherently lend themselves to lower power consumption due to their event-driven and sparse computational nature.

Furthermore, scaling brain-inspired models could lead to substantial improvements in performance for tasks where biological intelligence excels, such as complex pattern recognition, sensory processing, real-time decision-making in dynamic environments, and robust generalization from limited data (17).

These advancements could unlock new possibilities in various application domains, including medical diagnostics (for example, early detection of diseases through subtle changes in biological signals),

robotics (for example, more adaptable and dexterous robots for complex tasks),

and edge computing (for example, efficient AI on low-power devices for personalized assistance and environmental monitoring) (9).

The development of more adaptable and robust AI systems that can learn continuously and handle noisy or incomplete data more effectively is another key opportunity that could arise from successful scaling of brain-inspired models (16).

● Performance and Efficiency of Brain-Inspired Foundation Models Compared to Traditional Architectures:

○ Performance Comparison:

The performance of brain-inspired AI models in comparison to traditional architectures, particularly Transformer-based foundation models, is an area of active investigation.

While traditional deep learning models have achieved state-of-the-art results in many benchmarks, brain-inspired approaches show promise in certain domains (36).

For example, MovieNet, inspired by the brain's visual processing, has demonstrated superior accuracy in video analysis compared to traditional models like GoogLeNet (27).

In tasks requiring adaptability, generalization from limited data, and robustness, brain-inspired AI often exhibits advantages (16).

However, for tasks where high precision on large, well-annotated datasets is crucial, traditional deep learning models may still hold an edge (2).

The choice of architecture often depends on the specific requirements of the application.

○ Efficiency Comparison:

One of the most compelling potential benefits of brain-inspired AI, especially neuromorphic computing and spiking neural networks, is their capacity for significantly greater energy efficiency compared to traditional artificial neural networks and foundation models (9).

The event-driven nature of SNNs, where neurons only compute and communicate when necessary, leads to sparse activity and reduced power consumption (10).

This inherent efficiency makes brain-inspired approaches particularly attractive for deployment in resource-constrained environments and for applications requiring real-time processing with limited power budgets.

○ Benchmarking Brain-Inspired AI (BIAI):

Evaluating the true capabilities of brain-inspired AI models requires benchmarks that go beyond the standard metrics used for traditional AI (16).

While accuracy on tasks like image classification and language understanding is important, benchmarks for BIAI should also assess aspects such as adaptability to novel situations, generalization from small datasets, robustness to noise and perturbations, learning speed, and even the degree of biological plausibility (117).

The development of such specialized benchmarks is crucial for providing a comprehensive understanding of the strengths and weaknesses of brain-inspired approaches and for guiding future research and development in the field.

● Future Directions and Research Areas in Brain-Inspired AI Foundation Models:

○ Hybrid Architectures:

A promising avenue for future research lies in the development of hybrid AI architectures that integrate brain-inspired algorithms, such as Spiking Neural Networks (SNNs) and Hierarchical Temporal Memory (HTM), with traditional deep learning foundation models (116).

By combining the strengths of both approaches, such as the scalability and generalizability of foundation models with the energy efficiency and biological plausibility of brain-inspired methods, researchers aim to overcome the limitations of each and create more powerful and versatile AI systems (116).

For instance, integrating SNNs with deep learning layers could enhance the processing of temporal data like Electroencephalography (EEG) signals, where traditional foundation models have faced challenges (116).

○ 11.2 Focus on Biological Plausibility:

Future research will likely continue to emphasize creating AI models that more closely mimic the structural and functional properties of specific brain regions and neural circuits (14).

This includes incorporating principles of neural connectivity, learning mechanisms observed in the brain (e.g., Hebbian learning), and the temporal dynamics of biological neural networks (16).

By increasing the biological plausibility of AI models, researchers hope to develop systems that are not only more efficient and adaptable but also provide valuable insights into the fundamental principles of intelligence in both artificial and biological systems.

○ Advancements in Neuromorphic Computing:

The continued advancement of neuromorphic computing is crucial for the future of brain-inspired AI foundation models (9).

This includes the development of more sophisticated and scalable neuromorphic hardware with increased numbers of neurons and synapses, as well as improved energy efficiency (9).

The creation of more user-friendly programming languages, development tools, and standardized frameworks for these specialized computing platforms will also be essential to broaden their accessibility and accelerate research (11).

Furthermore, the exploration of novel materials and device technologies for neuromorphic chips holds the promise of creating hardware that can more accurately and efficiently emulate the behavior of biological neurons and synapses (9).

○ Research into Brain Activity Data:

Future research will likely see an increased focus on leveraging the rich datasets of brain activity data (For example, Functional Magnetic Resonance Imaging (fMRI), Magnetoencephalography (MEG), Electroencephalography (EEG) ) that are becoming available (23).

These data can be used to train and evaluate brain-inspired AI models, helping to ensure that they are learning representations and processes that are relevant to biological intelligence.

The development of foundational models specifically for brain activity data, capable of learning meaningful representations from large-scale neuroimaging datasets, is also a promising direction (23).

Additionally, AI techniques themselves can be used as powerful tools to analyze and understand the complex patterns and dynamics of brain function, fostering a synergistic relationship between the fields of AI and neuroscience.

○ Addressing Ethical and Societal Implications:

As brain-inspired AI foundation models advance and potentially become more capable and human-like, it will be increasingly important to address the ethical and societal implications that arise (21).

This includes focusing on the transparency and interpretability of these models to understand how they make decisions, ensuring accountability for their actions, and mitigating potential biases that might be learned from training data or inherited from our understanding of the brain (6).

Responsible development and deployment of brain-inspired AI will require careful consideration of these ethical issues to ensure that these technologies are safe, fair, and beneficial for society.

● Conclusion:

The Potential of Brain-Inspired Architectures in Shaping the Future of AI Foundation Models:

○ Summary of Key Findings:

Emerging alternative AI foundation model architectures, inspired by the intricate structure and function of specific brain regions, hold considerable potential to address the limitations inherent in current dominant Transformer-based models.

By drawing inspiration from the specialized processing capabilities of the visual cortex, auditory cortex, hippocampus, language-related areas, and the general learning principles of the neocortex, researchers are actively developing specialized AI models tailored for a diverse range of tasks. Approaches such as neuromorphic computing, spiking neural networks, hierarchical temporal memory, and graph-based models represent the forefront of this movement, each offering unique advantages in terms of computational efficiency, adaptability to novel situations, and the capacity to effectively handle various types of data.

○ Future Outlook:

The trajectory of AI foundation models is increasingly intertwined with the insights gained from neuroscience.

The foreseeable future promises a deepening convergence between these two fields, with ongoing research poised to unlock significant breakthroughs in AI capabilities.

Hybrid architectures, seamlessly integrating the strengths of brain-inspired algorithms with the scalability and generalizability of traditional deep learning frameworks, stand out as a particularly promising direction.

Furthermore, continued advancements in neuromorphic computing hardware, coupled with a more profound understanding of brain activity data, are expected to play crucial roles in enabling the development of more powerful, efficient, and biologically plausible AI systems.

○ Concluding Remarks:

Brain-inspired architectures represent a vital and exciting frontier in the ongoing evolution of artificial intelligence.

While substantial challenges remain in scaling these models to the levels of their traditional counterparts and in fully replicating the complexities of the human brain, the potential benefits are immense.

These include the promise of more sustainable and energy-efficient AI, enhanced performance in specialized tasks where biological intelligence excels, and the development of more adaptable and robust intelligent systems.

As research in this interdisciplinary field continues to progress, these alternative approaches have the capacity to fundamentally reshape the landscape of AI foundation models, paving the way for a future where artificial intelligence is not only powerful but also more aligned with the principles of natural intelligence.

Works Cited:

1. The Modern AI Stack: Design Principles for the Future of Enterprise AI Architectures, accessed March 18, 2025, https://menlovc.com/perspective/the-modern-ai-stack-design-principles-for-the-future-of-enterprise-ai-architectures/

2. Foundation Models vs. Traditional AI: What's the Difference? | by ..., accessed March 18, 2025, https://medium.com/@hakeemsyd/foundation-models-vs-traditional-ai-whats-the-difference-ad9f3f097dec

3. What Are Foundation Models in AI? - DataCamp, accessed March 18, 2025, https://www.datacamp.com/blog/introduction-to-foundation-models

4. What are Foundation Models? - Generative AI - AWS, accessed March 18, 2025, https://aws.amazon.com/what-is/foundation-models/

5. The Foundation Model: Key Facts and Insights | BasicAI's Blog, accessed March 18, 2025, https://www.basic.ai/blog-post/what-is-the-foundation-model

6. Limitations of Transformer Architecture | by Thirupathi Thangavel ..., accessed March 18, 2025, https://medium.com/@thirupathi.thangavel/limitations-of-transformer-architecture-4e6118cbf5a4

7. On Limitations of the Transformer Architecture - arXiv, accessed March 18, 2025, https://arxiv.org/html/2402.08164v1

8. On Limitations of the Transformer Architecture - OpenReview, accessed March 18, 2025, https://openreview.net/forum?id=KidynPuLNW

9. What Is Neuromorphic Computing? - IBM, accessed March 18, 2025, https://www.ibm.com/think/topics/neuromorphic-computing

10. Neuromorphic Engineering: Developing Brain-Inspired Machines ..., accessed March 18, 2025, https://viso.ai/deep-learning/neuromorphic-engineering/

11. Scaling up Neuromorphic Computing for More Efficient and Effective AI Everywhere and Anytime, accessed March 18, 2025, https://today.ucsd.edu/story/scaling-up-neuromorphic-computing-for-more-efficient-and-effective-ai-everywhere-and-anytime

12. A Survey on Neuromorphic Architectures for Running Artificial Intelligence Algorithms, accessed March 18, 2025, https://www.mdpi.com/2079-9292/13/15/2963

13. Neuromorphic engineering: Artificial brains for artificial intelligence - PMC - PubMed Central, accessed March 18, 2025, https://pmc.ncbi.nlm.nih.gov/articles/PMC11668493/

14. Advancing brain-inspired computing with hybrid neural networks - Oxford Academic, accessed March 18, 2025, https://academic.oup.com/nsr/article/11/5/nwae066/7614610

15. Brain-Inspired Machines: The Neural Networks Shaping AI's Future - The Greeley Voice, accessed March 18, 2025, https://thegreeleyvoice.com/2483/science-technology/brain-inspired-machines-the-neural-networks-shaping-ais-future/

16. Brain-inspired Artificial Intelligence: A Comprehensive Review - arXiv, accessed March 18, 2025, https://arxiv.org/pdf/2408.14811

17. Brain-inspired Artificial Intelligence: A Comprehensive Review - Powerdrill, accessed March 18, 2025, https://powerdrill.ai/discover/discover-Brain-inspired-Artificial-Intelligence-cm0ecc8t1gahz01b21papf451

18. Brain-Inspired AI: A New Horizon in Cybersecurity - Azar Alili, accessed March 18, 2025, https://azaralili.com/f/brain-inspired-ai-a-new-horizon-in-cybersecurity

19. Brain Inspired AI: A Deep Dive on Neuromorphic Computing - Medium, accessed March 18, 2025, https://medium.com/garden-research/brain-inspired-ai-a-deep-dive-on-neuromorphic-computing-39ff5d995f45

20. The Benefits of Brain-Inspired Computing | Innovate - IEEE Xplore, accessed March 18, 2025, https://innovate.ieee.org/innovation-spotlight/the-benefits-of-brain-inspired-computing/

21. Advantages And Disadvantages Of Biologically-Inspired AI | Restackio, accessed March 18, 2025, https://www.restack.io/p/biologically-inspired-ai-answer-advantages-disadvantages-cat-ai

22. Scaling a Brain-Inspired Approach to Computing - Pitt Swanson School of Engineering, accessed March 18, 2025, https://news.engineering.pitt.edu/scaling-a-brain-inspired-approach-to-computing/

23. Towards a synergy between AI and the brain: Microsoft Research Asia StarTrack Scholars 2025 explores the frontiers of AI and neuroscience with you, accessed March 18, 2025, https://www.microsoft.com/en-us/research/articles/towards-a-synergy-between-ai-and-the-brain-microsoft-research-asia-startrack-scholars-2025-explores-the-frontiers-of-ai-and-brain-science-with-you/

24. Top 7 Trends In Neuromorphic Computing - Verified Market Reports [2025], accessed March 18, 2025, https://www.verifiedmarketreports.com/blog/top-7-trends-in-neuromorphic-computing/

25. Neuromorphic Computing Advancements 2024 - FirstIgnite, accessed March 18, 2025, https://firstignite.com/exploring-the-latest-neuromorphic-computing-advancements-in-2024/

26. Exploring the potential of neuromorphic computing | Scilight - AIP Publishing, accessed March 18, 2025, https://pubs.aip.org/aip/sci/article/2025/2/021103/3329786/Exploring-the-potential-of-neuromorphic-computing

27. Brain-Inspired AI Learns to Watch Videos Like a Human ..., accessed March 18, 2025, https://neurosciencenews.com/brain-ai-video-watching-28218/

28. Neuroanatomy, Visual Cortex - StatPearls - NCBI Bookshelf, accessed March 18, 2025, https://www.ncbi.nlm.nih.gov/books/NBK482504/

29. The Auditory Cortex - Neuroscience - NCBI Bookshelf, accessed March 18, 2025, https://www.ncbi.nlm.nih.gov/books/NBK10900/

30. Learning, Recalling, and Thinking - Discovering the Brain - NCBI Bookshelf, accessed March 18, 2025, https://www.ncbi.nlm.nih.gov/books/NBK234153/

31. Speech & Language | Memory and Aging Center - University of California, San Francisco, accessed March 18, 2025, https://memory.ucsf.edu/symptoms/speech-language

32. Temporal-Sequential Learning With a Brain-Inspired ... - Frontiers, accessed March 18, 2025, https://www.frontiersin.org/journals/computational-neuroscience/articles/10.3389/fncom.2020.00051/full

33. Functional specificity in the human brain: A window into the functional architecture of the mind | PNAS, accessed March 18, 2025, https://www.pnas.org/doi/10.1073/pnas.1005062107

34. Visual Cortex - Vivid Vision, accessed March 18, 2025, https://www.seevividly.com/info/Physiology_of_Vision/The_Brain/Visual_System/Visual_Cortex

35. Visual Processing: Cortical Pathways (Section 2, Chapter 15 ..., accessed March 18, 2025, https://nba.uth.tmc.edu/neuroscience/m/s2/chapter15.html

36. Brain-inspired Artificial Intelligence: A Comprehensive Review - arXiv, accessed March 18, 2025, https://arxiv.org/html/2408.14811v1

37. Cognitive Neuroscience and AI: Unlocking the Future of Intelligence - Open MedScience, accessed March 18, 2025, https://openmedscience.com/cognitive-neuroscience-and-ai-unlocking-the-future-of-intelligence/

38. Visual Perception Inspired Deep Learning Models: A Study of Visual Attention and Information Interaction - IEEE Computer Society, accessed March 18, 2025, https://www.computer.org/csdl/proceedings-article/mvipit/2024/430300a066/253bZZI2y6Q

39. neurosciencenews.com, accessed March 18, 2025, https://neurosciencenews.com/brain-ai-video-watching-28218/#:~:text=Summary%3A%20Researchers%20have%20developed%20MovieNet,moving%20images%20with%20unprecedented%20accuracy.

40. Scripps Research scientists create AI that “watches” videos by ..., accessed March 18, 2025, https://www.scripps.edu/news-and-events/press-room/2024/20241209-cline-movienet.html

41. Scripps Research develops brain-inspired AI for real-time video analysis - News-Medical, accessed March 18, 2025, https://www.news-medical.net/news/20241209/Scripps-Research-develops-brain-inspired-AI-for-real-time-video-analysis.aspx

42. serre-lab.clps.brown.edu, accessed March 18, 2025, https://serre-lab.clps.brown.edu/wp-content/uploads/2012/08/serre_etal-CVPR05.pdf

43. (PDF) Object Recognition with Features Inspired by Visual Cortex - ResearchGate, accessed March 18, 2025, https://www.researchgate.net/publication/4156337_Object_Recognition_with_Features_Inspired_by_Visual_Cortex

44. High-performing neural network models of visual cortex benefit from ..., accessed March 18, 2025, https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1011792

45. Computational Model of Primary Visual Cortex Combining Visual ..., accessed March 18, 2025, https://pmc.ncbi.nlm.nih.gov/articles/PMC4489578/

46. Computational models of visual attention - Scholarpedia, accessed March 18, 2025, https://www.scholarpedia.org/article/Computational_models_of_visual_attention

47. Bridging Neuroscience and AI: Insights From the Retina and Beyond - Snowflake, accessed March 18, 2025, https://www.snowflake.com/engineering-blog/bridging-neuroscience-ai-retina/?lang=es&utm_cta=website-data-cloud-tour-americas-hub

48. Hierarchical Temporal Memory: Overview | by Srishti Mishra - Medium, accessed March 18, 2025, https://medium.com/hierarchical-learning/hierarchical-temporal-memory-overview-d411c9e4f90e

49. Auditory cortex - Wikipedia, accessed March 18, 2025, https://en.wikipedia.org/wiki/Auditory_cortex

50. Function and Association of Auditory Cortex with Brain - Longdom Publishing SL, accessed March 18, 2025, https://www.longdom.org/open-access/function-and-association-of-auditory-cortex-with-brain-99342.html

51. New study reveals how brain changes when you learn a sound, accessed March 18, 2025, https://www.bme.jhu.edu/news-events/news/new-study-reveals-how-brain-changes-when-you-learn-a-sound/

52. Computational Model for Auditory Cortex: An Analogy to Visual Cortex, accessed March 18, 2025, https://www.kecl.ntt.co.jp/people/terashima.hiroki/paper/terashima2014thesis.pdf

53. Dissecting neural computations in the human auditory pathway using deep neural networks for speech - PubMed Central, accessed March 18, 2025, https://pmc.ncbi.nlm.nih.gov/articles/PMC10689246/

54. Parallel hierarchical encoding of linguistic representations in the ..., accessed March 18, 2025, https://www.biorxiv.org/content/10.1101/2025.01.30.635775v1

55. Parallel hierarchical encoding of linguistic representations in the human auditory cortex and recurrent automatic speech recognition systems | bioRxiv, accessed March 18, 2025, https://www.biorxiv.org/content/10.1101/2025.01.30.635775v1.full-text

56. An Audio-Visual Speech Separation Model Inspired by Cortico-Thalamo-Cortical Circuits, accessed March 18, 2025, https://www.computer.org/csdl/journal/tp/2024/10/10487874/1VORkMV7PCE

57. Computational Neural Modeling of Auditory Cortical Receptive Fields - Frontiers, accessed March 18, 2025, https://www.frontiersin.org/journals/computational-neuroscience/articles/10.3389/fncom.2019.00028/full

58. Deep neural networks effectively model neural adaptation to changing background noise and suggest nonlinear noise filtering methods in auditory cortex, accessed March 18, 2025, https://pmc.ncbi.nlm.nih.gov/articles/PMC10510744/

59. Where are memories stored in the brain? - Queensland Brain Institute, accessed March 18, 2025, https://qbi.uq.edu.au/memory/where-are-memories-stored

60. The Neuroscience of Memory: How the Brain Stores and Retrieves Information, accessed March 18, 2025, https://www.openaccessjournals.com/articles/the-neuroscience-of-memory-how-the-brain-stores-and-retrieves-information-18236.html

61. Neuroanatomy of memory - Wikipedia, accessed March 18, 2025, https://en.wikipedia.org/wiki/Neuroanatomy_of_memory

62. A Brain-Inspired Model of Hippocampal Spatial Cognition Based on a Memory-Replay Mechanism - MDPI, accessed March 18, 2025, https://www.mdpi.com/2076-3425/12/9/1176

63. Parts of the Brain Involved with Memory – Psychology - UH Pressbooks, accessed March 18, 2025, https://pressbooks-dev.oer.hawaii.edu/psychology/chapter/parts-of-the-brain-involved-with-memory/

64. A model of autonomous interactions between hippocampus and neocortex driving sleep-dependent memory consolidation - PMC - PubMed Central, accessed March 18, 2025, https://pmc.ncbi.nlm.nih.gov/articles/PMC9636926/

65. Cortico-Hippocampal Computational Modeling Using Quantum-Inspired Neural Networks, accessed March 18, 2025, https://www.frontiersin.org/journals/computational-neuroscience/articles/10.3389/fncom.2020.00080/full

66. A Computational Model for Episodic Memory Inspired by the Brain by Adam Peter Trischler, accessed March 18, 2025, https://utoronto.scholaris.ca/server/api/core/bitstreams/6fe410e1-d3c4-4e28-b58a-8a7eaa34959d/content

67. A Brain-Inspired Model of Hippocampal Spatial Cognition Based on ..., accessed March 18, 2025, https://pmc.ncbi.nlm.nih.gov/articles/PMC9496859/

68. Computational models of hippocampal functions, accessed March 18, 2025, https://www.oxcns.org/papers/573%20Rolls%202017%20Computational%20Models%20of%20Hippocampal%20Functions.pdf

69. A dynamical computational model of theta generation in hippocampal circuits to study theta-gamma oscillations during neurostimulation - eLife, accessed March 18, 2025, https://elifesciences.org/reviewed-preprints/87356

70. (PDF) A Survey of Trends in Hippocampal-Formation Inspired ..., accessed March 18, 2025, https://www.researchgate.net/publication/386094451_A_Survey_of_Trends_in_Hippocampal-Formation_Inspired_Computational_Models

71. AI's memory-forming mechanism found to be strikingly similar to that of the brain, accessed March 18, 2025, https://www.sciencedaily.com/releases/2023/12/231218130031.htm

72. In a Striking Discovery, AI Shows Human-Like Memory Formation - SciTechDaily, accessed March 18, 2025, https://scitechdaily.com/in-a-striking-discovery-ai-shows-human-like-memory-formation/

73. Neuroplasticity Meets Artificial Intelligence: A Hippocampus-Inspired ..., accessed March 18, 2025, https://pmc.ncbi.nlm.nih.gov/articles/PMC11591613/

74. Improved Hierarchical Temporal Memory for Online Prediction of Ocean Time Series Data, accessed March 18, 2025, https://www.mdpi.com/2077-1312/12/4/574

75. Brain Areas That Control Language And Speech - Language - MCAT Content - Jack Westin, accessed March 18, 2025, https://jackwestin.com/resources/mcat-content/language/brain-areas-that-control-language-and-speech

76. The Foundations of Language Processing - BrainFacts, accessed March 18, 2025, https://www.brainfacts.org/thinking-sensing-and-behaving/language/2012/language-processing

77. Language processing in the brain | Intro to Brain and Behavior Class Notes | Fiveable, accessed March 18, 2025, https://library.fiveable.me/introduction-brain-behavior/unit-9/language-processing-brain/study-guide/kgma0yX4Xr2VFMaj

78. 15.2: Broca's Area, Wernicke's Area, and Additional Language-Processing Areas in the Brain - Social Sci LibreTexts, accessed March 18, 2025, https://socialsci.libretexts.org/Bookshelves/Psychology/Biological_Psychology/Biopsychology_(OERI)_-_DRAFT_for_Review/15%3A_Language_and_the_Brain/15.02%3A_Broca's_Area_Wernicke's_Area_and_Additional_Language-Processing_Areas_in_the_Brain

79. From Sound to Meaning: Navigating Wernicke's Area in Language Processing - PMC, accessed March 18, 2025, https://pmc.ncbi.nlm.nih.gov/articles/PMC11491986/

80. On Architectures for Natural Language Processing - John Sowa, accessed March 18, 2025, https://www.jfsowa.com/talks/bionlp.pdf

81. Biological and Psycholinguistic Influences On Architectures for Natural Language Processing - CiteSeerX, accessed March 18, 2025, https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=db8c4f9dcf150499fe39e32187b607bb0b608555

82. The Brain Basis of Language Processing: From Structure to Function, accessed March 18, 2025, https://journals.physiology.org/doi/full/10.1152/physrev.00006.2011

83. From Sound to Meaning: Navigating Wernicke's Area in Language Processing | Cureus, accessed March 18, 2025, https://www.cureus.com/articles/283534-from-sound-to-meaning-navigating-wernickes-area-in-language-processing.pdf

84. Emergent modularity in large language models: Insights from ..., accessed March 18, 2025, https://www.biorxiv.org/content/10.1101/2025.02.22.639416v1.full-text

85. Neurolinguistic Computational Models - PsyLing, accessed March 18, 2025, https://psyling.talkbank.org/years/2008/stemmer.pdf

86. A Biologically Inspired Computational Model of Language Cognition - CiteSeerX, accessed March 18, 2025, https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=e67ab465a435a86f4fe4b51347948b04e5211fb3

87. Brain-inspired AI Agent: The Way Towards AGI - arXiv, accessed March 18, 2025, https://arxiv.org/html/2412.08875v1

88. hierarchical temporal memory - Laboratory of Brain, Hearing and Behavior, accessed March 18, 2025, https://hearingbrain.org/docs/HTM_white_paper.pdf

89. A Machine Learning Guide to HTM (Hierarchical Temporal Memory) - Numenta, accessed March 18, 2025, https://www.numenta.com/blog/2019/10/24/machine-learning-guide-to-htm/

90. Hierarchical Temporal Memory - Dileep George, accessed March 18, 2025, https://dileeplearning.github.io/uploads/HTM.pdf

91. NeoSLAM: Long-Term SLAM Using Computational Models of the ..., accessed March 18, 2025, https://www.mdpi.com/1424-8220/24/4/1143

92. NILRNN: A Neocortex-Inspired Locally Recurrent Neural Network for Unsupervised Feature Learning in Sequential Data - Semantic Scholar, accessed March 18, 2025, https://www.semanticscholar.org/paper/NILRNN%3A-A-Neocortex-Inspired-Locally-Recurrent-for-Van-Horenbeke-Peer/d4065f5b5fa1968dc53cc9b17e264e17a1ca0be2

93. (PDF) Scalable Frameworks for Hierarchical Temporal Memory in ..., accessed March 18, 2025, https://www.researchgate.net/publication/387937579_Scalable_Frameworks_for_Hierarchical_Temporal_Memory_in_Processing_Distributed_and_Dynamic_Data_Stream

94. The Use of Hierarchical Temporal Memory and Temporal Sequence Encoder for Online Anomaly Detection in Industrial Cyber-Physical Systems - MDPI, accessed March 18, 2025, https://www.mdpi.com/2073-4441/17/3/321

95. A New Hierarchical Temporal Memory Algorithm Based on Activation Intensity - PMC, accessed March 18, 2025, https://pmc.ncbi.nlm.nih.gov/articles/PMC8803450/

96. An Overview of the Hierarchical Temporal Memory Accelerators - ResearchGate, accessed March 18, 2025, https://www.researchgate.net/publication/364310661_An_Overview_of_the_Hierarchical_Temporal_Memory_Accelerators

97. (PDF) NILRNN: A Neocortex-Inspired Locally Recurrent Neural Network for Unsupervised Feature Learning in Sequential Data - ResearchGate, accessed March 18, 2025, https://www.researchgate.net/publication/368753713_NILRNN_A_Neocortex-Inspired_Locally_Recurrent_Neural_Network_for_Unsupervised_Feature_Learning_in_Sequential_Data

98. Dynamic predictive coding: A model of hierarchical sequence learning and prediction in the neocortex | PLOS Computational Biology, accessed March 18, 2025, https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1011801

99. Towards Graph Foundation Models - The Web Conference 2024, accessed March 18, 2025, https://www2024.thewebconf.org/docs/tutorial-slides/towards-graph-foundation-models.pdf

100. Towards Graph Foundation Models - Chuan Shi, accessed March 18, 2025, https://shichuan.org/doc/WWW24Tutorial_GFM_Part1.pdf

101. arxiv.org, accessed March 18, 2025, https://arxiv.org/pdf/2310.11829

102. [2402.02216] Position: Graph Foundation Models are Already Here - arXiv, accessed March 18, 2025, https://arxiv.org/abs/2402.02216

103. Spiking Neural Networks: The next “Big Thing” in AI? | by Dean S Horak | Medium, accessed March 18, 2025, https://medium.com/@deanshorak/spiking-neural-networks-the-next-big-thing-in-ai-efe3310709b0

104. Toward Large-scale Spiking Neural Networks: A Comprehensive Survey and Future Directions - arXiv, accessed March 18, 2025, https://arxiv.org/html/2409.02111v1

105. Brain Inspired AI Learns Like Humans - Neuroscience News, accessed March 18, 2025, https://neurosciencenews.com/neuroai-learning-anns-26349/

106. Efficient parameter calibration and real-time simulation of large-scale spiking neural networks with GeNN and NEST - Frontiers, accessed March 18, 2025, https://www.frontiersin.org/journals/neuroinformatics/articles/10.3389/fninf.2023.941696/full

107. www.biorxiv.org, accessed March 18, 2025, https://www.biorxiv.org/content/10.1101/2024.09.24.614728v1.full.pdf

108. Efficient Simulation of Large-Scale Spiking Neural Networks Using CUDA Graphics Processors - Donald Bren School of Information and Computer Sciences, accessed March 18, 2025, https://ics.uci.edu/~jmoorkan/pub/gpusnn-ijcnn.pdf

109. Adaptive Synaptic Scaling in Spiking Networks for Continual Learning and Enhanced Robustness - PubMed, accessed March 18, 2025, https://pubmed.ncbi.nlm.nih.gov/38536699/

110. Spiking Neural Networks and Their Applications: A Review - PMC - PubMed Central, accessed March 18, 2025, https://pmc.ncbi.nlm.nih.gov/articles/PMC9313413/

111. On the Future of Training Spiking Neural Networks, accessed March 18, 2025, https://www.dfki.de/fileadmin/user_upload/import/12987_ICPRAM_2023_118_CR.pdf

112. On the Future of Training Spiking Neural Networks - SciTePress, accessed March 18, 2025, https://www.scitepress.org/Papers/2023/117455/117455.pdf

113. Biologically-Inspired AI Benefits And Disadvantages - Restack, accessed March 18, 2025, https://www.restack.io/p/biologically-inspired-ai-answer-benefits-disadvantages-cat-ai

114. In Nature article, researchers chart course for scalable, neuromorphic computing systems, accessed March 18, 2025, https://www.royalholloway.ac.uk/research-and-education/departments-and-schools/computer-science/news/in-nature-article-researchers-chart-course-for-scalable-neuromorphic-computing-systems/

115. Neuroscience-Inspired Artificial Intelligence | Johns Hopkins University Applied Physics Laboratory, accessed March 18, 2025, https://www.jhuapl.edu/work/projects-and-missions/neuroscience-inspired-artificial-intelligence

116. Toward brain-inspired foundation model for EEG signal processing: our opinion - PMC, accessed March 18, 2025, https://pmc.ncbi.nlm.nih.gov/articles/PMC11652537/

117. Biologically-Inspired AI Benchmarks | Restackio, accessed March 18, 2025, https://www.restack.io/p/biologically-inspired-ai-answer-benchmarks-cat-ai

118. Benchmarking Predictive Coding Networks Made Simple - Verses AI, accessed March 18, 2025, https://www.verses.ai/research-blog/benchmarking-predictive-coding-networks-made-simple

119. Toward brain-inspired foundation model for EEG signal processing: our opinion - Frontiers, accessed March 18, 2025, https://www.frontiersin.org/journals/neuroscience/articles/10.3389/fnins.2024.1507654/full

120. Brain-inspired Computational Modeling of Action Recognition with Recurrent Spiking Neural Networks Equipped with Reinforcement Delay Learning - arXiv, accessed March 18, 2025, https://arxiv.org/html/2406.11778v1

121. Innovations in AI: Brain-inspired design for more capable and sustainable technology - Microsoft Research, accessed March 18, 2025, https://www.microsoft.com/en-us/research/blog/innovations-in-ai-brain-inspired-design-for-more-capable-and-sustainable-technology/

122. Shifting Attention to You: Personalized Brain-Inspired AI Models - arXiv, accessed March 18, 2025, https://arxiv.org/html/2502.04658v1

123. Global Neuromorphic Computing and Sensing Market 2025-2035: In-depth Analysis of Market Trends, Technological Advancements and Growth Ppportunities in this Rapidly Evolving Sector - GlobeNewswire, accessed March 18, 2025, https://www.globenewswire.com/news-release/2025/02/17/3027128/0/en/Global-Neuromorphic-Computing-and-Sensing-Market-2025-2035-In-depth-Analysis-of-Market-Trends-Technological-Advancements-and-Growth-Ppportunities-in-this-Rapidly-Evolving-Sector.html

124. In Nature article, UTSA researchers chart course for scalable ..., accessed March 18, 2025, https://www.utsa.edu/today/2025/01/story/nature-article-neuromorphic-computing-systems.html

125. Brain-Inspired Technologies that is advancing AI | by Joe El Khoury - Medium, accessed March 18, 2025, https://medium.com/@jelkhoury880/brain-inspired-technologies-that-is-advancing-ai-6bd2bd76b540

"Any questions?"

\

??????