Brain-Inspired Approaches: Charting the Future of Academic AI Research Centers

The Artificial Intelligence (AI) sector is experiencing record investment. Projections indicate that global AI investment is set to surpass $12 billion in 2024, marking a significant increase from previous years. This trend is already evident, with $3 billion invested in AI during the first quarter of 2024 alone, signaling a robust start to the year and suggesting that the overall projection may even be conservative. A substantial portion of this investment is being channeled into computational infrastructure and research, reflecting a shift in focus towards "AI infrastructure," as noted in the Stanford AI Index report. This shift is particularly noteworthy as it indicates a growing recognition of the importance of foundational elements in AI development rather than just end-user applications. Companies developing core AI applications are attracting significant investments, highlighting the industry's focus on pushing the boundaries of AI capabilities.

Generative AI has emerged as a particularly attractive area for investors, capturing the imagination of both the tech industry and the general public. In 2023, it attracted a staggering $25.2 billion globally, nearly nine times the investment seen in 2022. This exponential growth reflects the widespread excitement about the potential applications of generative AI across various domains, from content creation to scientific research. The trend is expected to continue into 2024, with generative AI likely to remain a hot spot for investment and innovation. Similarly, investment in AI core technologies, including large language models and semiconductors, has seen exponential growth, rising from a modest $77 million in 2018 to a substantial $5.1 billion in 2023. This dramatic increase underscores the growing recognition of the critical role that these foundational technologies play in advancing the field of AI. The focus on semiconductors highlights the importance of hardware advancements in enabling more powerful and efficient AI systems.

The costs associated with training state-of-the-art AI models have reached unprecedented levels, reflecting the immense computational resources required to develop cutting-edge AI systems. For instance, OpenAI's GPT-4, one of the most advanced language models to date, required an estimated $78 million worth of compute to train. Even more staggering is the case of Google's Gemini Ultra, which reportedly cost $191 million in computational resources alone. These figures underscore the substantial investment being made in computational infrastructure and highlight the increasing barriers to entry for AI research and development. However, despite these remarkable advances and the substantial resources being poured into AI development, there are indications that the reasoning capabilities of large language models may be approaching a plateau. This suggests that simply scaling up model size may not be the most effective path to more sophisticated AI systems. Researchers are beginning to question whether the current approach of training ever-larger models on ever-larger datasets will continue to yield proportional improvements in AI capabilities.

Moreover, the energy demands of training and operating these large models raise significant environmental concerns, calling into question the sustainability of current AI development practices. Training a single, relatively small BERT model emits approximately 652 kg of CO2, comparable to a trans-American flight. For larger models like GPT, the environmental impact is even more striking, with training estimated to produce around 552 tons of CO2, equivalent to driving a car for 1.2 million miles. As models continue to grow in size and complexity, these energy demands and their associated carbon footprints become increasingly unsustainable, prompting calls for more energy-efficient approaches to AI development. In stark contrast to these energy-intensive AI models, the human brain operates remarkably efficiently, consuming only about 20 watts of power - equivalent to a dim light bulb - yet performing complex cognitive tasks that still challenge our most advanced AI systems. This biological efficiency suggests that there may be alternative approaches to AI development that could yield more sustainable and capable systems. The brain's ability to learn incrementally, building on prior knowledge and adapting to new information throughout life - a process known as continual learning - offers a compelling model for more efficient AI systems.

The role of academic AI centres in this age of generative AI

Academic institutions, despite their crucial role in advancing AI research, often face significant challenges in accessing and utilizing Generative AI, particularly large language models. The substantial computational requirements of these models, necessitating extensive hardware for training and deployment, can be financially and technologically beyond the reach of many academic institutions. This creates a potential innovation gap, where a lack of access to necessary resources may hinder groundbreaking ideas. Also, the centralized approach to AI development, necessitating high-performance computing infrastructure, creates barriers that may stifle innovation and limit diverse perspectives. The substantial investments needed for cutting-edge AI research can concentrate advancements in the hands of a few well-resourced entities, potentially narrowing the field of contributors to AI progress.

In response to these challenges, university research centers must explore alternative approaches to progress towards Artificial General Intelligence (AGI). These approaches draw inspiration from the workings of the human brain and its learning mechanisms, potentially leading to more efficient and capable AI systems. This shift aligns with recent research highlighting the promise of small language models, which offer the potential for more accessible and sustainable AI development, addressing some of the limitations inherent in larger models. Emerging solutions and research directions are shaping the future of AI development. One promising avenue is the development of edge-cloud collaboration frameworks for AI service provision. This approach combines large cloud models with smaller edge models, potentially offering a more distributed and efficient architecture for AI deployment. This approach could reduce latency, enhance privacy, and improve overall system efficiency by processing some data locally on edge devices and leveraging cloud resources for more complex tasks.

Another innovative direction involves using mathematical embedding and "Math Agents" to enhance AI capabilities. This approach aims to imbue AI systems with a more fundamental understanding of mathematical concepts, potentially leading to more rigorous and provable AI systems. Such advancements could be particularly useful in fields like genomics and health informatics, where precise and verifiable computations are crucial. The convergence of quantum computing and AI presents both exciting opportunities and formidable challenges. Quantum AI has the potential to solve complex problems that are currently intractable for classical computers, potentially leading to breakthroughs in areas such as drug discovery, financial modeling, and cryptography.

As generative AI models become more prevalent and powerful, there are growing concerns about their impact on the digital commons, including issues related to copyright, misinformation, and the potential for misuse. In response, research can be conducted on governance-based solutions, including standardized dataset and model disclosure practices, increased transparency in model training and capabilities, and structures for shared ownership based on data provision. These efforts will ensure that the benefits of AI are distributed fairly and that potential negative impacts are mitigated.

Other research directions that are gaining traction in the pursuit of brain-inspired AI approaches include neuromorphic computing, which focuses on developing hardware and algorithms that more closely mimic the structure and function of biological neural networks, potentially leading to more energy-efficient and adaptable AI systems. Continual learning explores methods for AI systems to learn incrementally and adapt to new information without catastrophic forgetting, addressing one of the key limitations of current AI models. Cognitive architectures aim to create comprehensive models of cognition that integrate perception, reasoning, and action, potentially leading to AI systems with more human-like intelligence. Embodied AI investigates the role of physical embodiment and sensorimotor experience in the development of intelligence, challenging the notion that intelligence can be fully achieved through disembodied computation alone. Lastly, causal reasoning seeks to develop AI systems capable of understanding and reasoning about cause-and-effect relationships, a crucial component of human-like intelligence that is still challenging for current AI systems.

By focusing on these brain-inspired approaches, university AI centers can contribute unique and valuable insights to the field of AI, complementing the large-scale empirical approaches favoured by industry. This shift could lead to more sustainable, efficient, and potentially more powerful AI systems that bring us closer to the goal of artificial general intelligence. Additionally, the emphasis on ethical considerations and governance frameworks will ensure the responsible development and deployment of AI technologies. The collaborative efforts of academia, industry, and policymakers will be crucial in navigating this complex landscape and ensuring that the development of AI benefits humanity as a whole