The Futility of Brute Force AGI: Why the Human Brain's Efficiency is Key

The pursuit of Artificial General Intelligence (AGI) has led to an arms race in computational resources, with tech giants investing heavily in GPUs and sprawling data centres to train and run large language models boasting trillions of parameters. However, this brute force approach fundamentally misunderstands the nature of intelligence and ignores the incredible efficiency of the human brain.

The human brain, the epitome of intelligence, operates on a mere 20 watts of power - less than a typical lightbulb. This is possible thanks to the brain's approximately 86 billion neurons and 100 trillion synapses, which form an intricate network that enables massively parallel processing and efficient information transfer. Synaptic plasticity allows for rapid learning and adaptation, with the brain constantly rewiring itself based on experience. In contrast, current large language models require enormous amounts of energy and data to train and run. GPT-3, for instance, has 175 billion parameters and was trained on 45TB of text data. In contrast, the GPT-4 model is rumored to have even more parameters, potentially reaching 100 trillion. This is highly inefficient compared to human learning, which can grasp complex concepts from just a few examples - a capability known as "few-shot learning." Moreover, the human brain seamlessly integrates multiple modalities - sight, sound, touch, etc. - to form a coherent understanding of the world. Current AI models, on the other hand, are mostly limited to narrow domains.

The key to AGI lies not in scaling up computational resources, but in understanding and emulating the brain's architectural and functional principles. This includes:

1. Sparse coding: The brain efficiently represents information using a small number of active neurons at any given time.

2. Modularity: The brain is organised into specialised regions that work together, enabling efficient processing and transfer learning.

3. Predictive coding: The brain constantly generates predictions about the world, allowing for efficient processing of sensory input and rapid adaptation to changes.

4. Embodied cognition: Intelligence is deeply intertwined with the physical body and its interactions with the environment.

To make real progress towards AGI, we need to shift our focus from brute force computation to developing algorithms and architectures that capture the efficiency and flexibility of the human brain. This will require close collaboration between neuroscientists, cognitive scientists, and AI researchers. The path to AGI is not paved with GPUs and exaflops, but with a deep understanding of the efficiency of the human brain. Let us not get lost in the big numbers and instead focus on the fundamental principles that make human intelligence so remarkable.

References:

1. Simon B. Laughlin, Terrence J. Sejnowski, Communication in Neuronal Networks, Science,1870-1874(2003).DOI:10.1126/science.1089662
2. Brown RE. Donald O. Hebb and the Organization of Behavior: 17?years in the writing. Mol Brain. 2020 Apr 6;13(1):55. doi: 10.1186/s13041-020-00567-8. PMID: 32252813; PMCID: PMC7137474.
3. Brenden M. Lake?et al., Human-level concept learning through probabilistic program indiction. Science 350,1332 -1338(2015).DOI:10.1126/science.aab3050
4. Ghazanfar AA, Schroeder CE. Is neocortex essentially multisensory? Trends Cogn Sci. 2006 Jun;10(6):278-85. doi: 10.1016/j.tics.2006.04.008. Epub 2006 May 18. PMID: 16713325.
5. Olshausen, B., Field, D. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996). https://doi.org/10.1038/381607a0
6. Bullmore E, Sporns O. Complex brain networks: graph theoretical analysis of structural and functional systems. Nat Rev Neurosci. 2009 Mar;10(3):186-98. doi: 10.1038/nrn2575. Epub 2009 Feb 4. Erratum in: Nat Rev Neurosci. 2009 Apr;10(4):312. PMID: 19190637.