The Emperor's New Transcendence
Three years ago, I blogged about AI research (here) and explained why I wasn’t concerned that it might, on its own merit, pose us an existential threat. Though the last architectural breakthrough in AI design was in 2015, the runaway success of generative AI in the creative arts has since reignited public interest in the field (I’ve written about the rise of GANs, GPTs and VAEs, and how they function, here). Naturally, concerns over the ethics and dangers of the technology have also increased, and calls to curb AI development are not uncommon. After all, just because you can make a thing, it doesn’t mean that you should; it is likely, however, that before long somebody will.
Fear and Fortitude
So should we be concerned? Truly, I don't think so. Nothing happening currently in DeepMind, OpenAI, Baidu or any of the other major players is getting any nearer to artificial general intelligence, nor is it likely to. Current generative models, powerful as they are, may represent a blind alley on the route to AGI, and the burst of momentum we’ve just seen in the field may shortly wane now that little further progress can be made in that direction. As Noam Chomsky remarked of ChatGPT, it’s a nice toy.
That is all a large language model like GPT4 can ever be. It has studied an enormous volume of public literature, and some not so public, and made a probabilistic model of which letters, syllables and words generally follow others. Then, with much supervised and unsupervised training, it has made a thematic analysis of these associations so that it can classify any new message sent to it. To construct responses, it seeks material that aligns with the themes and, using that material, iterates attempts to build sentences for an appropriate response, until it can fool itself that its answer satisfies the question and fits with all the dialogue so far. Upon each iteration, it keeps track of its position in a sentence; it minds dependencies on the words it has already written and implications for words and parts of words that it should write subsequently.
But it is still only applying statistics and curated patterns to construct sensible, useful answers. It has no idea what it is doing. It has no opinion about it. It has no opinion about anything, and no purpose. All that is happening is maths. Real artificial intelligence will have to do, and be, something quite different. You can safely allow a generative adversarial network, a variational autoencoder or any of the other existing neural network architectures to access the internet, learn by itself, and improve its own programming iteratively; it will still not achieve awareness, not become an AGI.
Risk and Responsibility
But are there still dangers? What are the risks really? Might we abruptly change direction in AI design towards something that really could achieve general intelligence? If we were successful at that and allowed a true AGI to reprogram itself iteratively, or even create new and better AGIs, could our silicon offspring suddenly surpass and overwhelm us? Or is the real danger much nearer to hand? One of relatively dumb AI systems like GPT4 being used by malicious people for malevolent ends, or of relatively dumb AI systems being used by relatively dumb people to do unwise things?
As Mustafa Suleyman, cofounder of DeepMind, has pointed out, any AI can be dangerous in the hands of careless or bad actors. A car is dangerous in the wrong hands but can only harm a few hundred people at a time. Could an AI have far more harmful impact? Potentially. Perhaps the most obviously dangerous application would be autonomous weapons, but even allowing a machine-learning algorithm to control the stock markets directly might be unwise, as we came close to seeing a few years ago. It’s the people that are the problem, not the AI itself. Who can control the system? What will it be used for? What objectives are set for it? What is its output allowed to manipulate? Like genetic sequencing, nano-construction kits, nuclear weapons and many other powerful technologies, AI systems could be highly dangerous under the control of evil or stupid people. But unfortunately we can all be a little evil or a little stupid some of the time.
So there is danger here, and every AI development should be preceded by an exhaustive risk assessment and failure mode effect analysis. This is something that the leaders in AI research take very seriously, but there are organisations that have shown interest in the technology for more dubious purposes than creating art or having a chat. This is why we need strong international regulation against, for example, the construction of armed autonomous drones. All of these things help – regulation, risk analysis, danger awareness – so is there still something to be concerned about? Very definitely. AI needs our focus and we should be lobbying our governments and employers to prioritise their preparedness.
But what about AIs becoming a threat in their own right? What about artificial intelligence becoming actually intelligent?
Sense and Singularity
As I said in my article One Hundred Lines of Code, I don’t believe the mind is anything magical or metaphysical. It doesn’t rely on higher dimensions, undiscovered elementary particles or quantum physics. It is true that low-energy photons are liberated continually in the dendrites and may persist for some time, and there is evidence that quantum tunnelling may be involved in the transport of potassium ions, which become delocalised as they pass through the neuronal cell membranes. But while these processes may help explain how action potentials can be transmitted between neurons far faster than previously thought, none of that alters what cognition is. The human mind is software, embodied in the patterns of charge distribution amongst its hundred trillion primary synapses and quadrillions of inter-dendritic ones. Some of that software is generated and amended as we learn and grow, but more than half of it is inherited, programmed by our genes to take shape by the time we are twenty years old.
My argument was that the millions of years of evolutionary R&D leading to the current manifestation of the biological mind isn’t going to be reproduced in AI research any time soon. The counter-position would be that if we allow an AI to learn, grow and improve itself, or even to build new and better AIs, it could achieve the same level of development in the tiniest fraction of that time, given the speed of electrical signals in silicon chips compared to biological neurons. That argument was formulated before we understood the mechanism mentioned above that allows charge to flow far faster in the cortex than previously thought, but it’s still a fair point: electricity travels along a copper wire at almost the speed of light. The suggestion popularised by Max Tegmark, Stephen Hawking, Verner Vinge and others is that an artificial general intelligence, without constraint, would, within hours of achieving human-level sophistication, surpass us to the extent that we could no longer comprehend its reasoning or goals. We might easily become inconsequential to it and unable to compete.
The Blind Matchmaker
I don’t believe any existing AI architecture could ever lead to that. Not only does it have no goal, in the true sense of the word, but it has no perception what a goal might be or why it should develop one. It has no motivation whatsoever. The thing that is missing, the huge gulf between us and our AI systems, is that in our minds we have a detailed model of ourselves and our environment, while they have none at all. Without a model of the world and your location in it, you are not conscious, not aware; without a model of yourself and your own mind inside your mind, you are not self-aware. You do not know that you are alive.
So the mind must mind itself, and to do that it must know its own mind and know that it has a mind to mind itself with. That’s why, for any of our current or as-yet architected artificial intelligence systems, fears of them posing any existential threat to us in the future are unfounded.
Intelligence in the sense of conscious entities, then, is very much a part of being alive. Without a body or some kind of physical presence that can be modelled, there is little point in having a mind. Asimov was spot-on when he envisaged the robot body being an inseparable part of any artificial intelligence (see the robot stories and novels).
Were we to switch direction, then, to an AI approach in which a neural network built a sophisticated multi-referenced model of itself and its surroundings and kept that model continually updated, we might perhaps finally be on the road to developing an AGI. So could that pose an existential threat? To answer that we have first to ask ourselves what exactly we’re afraid of. And our only experience of high-level conscious intelligence is with mammals, particularly with one another.
We’re afraid of an AGI becoming like us.
General Intelligence
In my earlier article on the human brain and AI research, I stepped over the important point that what AI is actually aiming to mimic is only the cerebral isocortex, the outermost structure of the brain, wrapping most of the older regions in a uniform four millimetres of exclusively grey matter and constructed of two hemispheres, joined only by a thick bundle of nerve axons known as the corpus callosum. Each hemisphere appears to have the same distinct areas, separated by deep striations into frontal, temporal, parietal and occipital lobes, named after the adjacent bones of the skull. The isocortex is a mammalian innovation that has been around for roughly a hundred and fifty million years, but in our lineage it has grown dramatically in size and that happened during only the last three million years, more than half of that probably in the last million. In modern humans the isocortex makes up ninety percent of the overall outer cortex and forty percent of the entire brain’s mass. This expansion was able to proceed so rapidly because it was the simplest of genetic tricks – the isocortex is almost completely uniform from one end to the other, and it is very easy for DNA to build more of the same thing (more on evolution, genetics and epigenetics can be found here). Whilst it’s true that the frontal lobes appear most engaged in decision making, the occipital in visual perception, Wernicke’s area in written and spoken language comprehension, and so on, it is also true that when one of these areas is badly damaged, for example due to stroke, another can gradually take over the function. So despite there being these obvious lobes or regions with apparently dedicated functions, the acts of seeing, feeling, reading, writing and decision making may all involve exactly the same mechanism in neural network terms. The isocortex is a unified organ that may well have only a single trick. The challenge for AI researchers is to work out what that trick is.
Other parts of the brain we share with reptiles. These ancient organs, including cerebellum, brain stem and hippocampus, are the route through which all sensory input reaches the isocortex and are the only means of communicating with the rest of the body to achieve any action or movement. They are responsible for basic control functions in breathing, eating, posture and homeostasis, and it is also these relics that make memories permanent, interpret emotionally the input from our senses and take impulsive actions in response, through both nervous and endocrine systems. Without the isocortex, we were completely functional as living beings, as successful as a crocodile or a frog, but we didn’t really know that we were. We were not self-aware.
The Fable and the Fishing Net
The isocortex is an attempt by evolution to build apparatus for more complex reasoning, planning and decision-making, and it achieved that by modelling the animal’s body internally and externally as well as every aspect of the environment that it learns from its senses. It represents all this in time by holding records of how that model was in the past and making predictions for how it will be in the future. In fact those predictions are all we ever experience. We are not consciously aware of anything coming in from our senses, only the hypotheses our isocortex is making about what might be out there and what might be happening next. The model is all we know, and it is a storytelling giant. The self is a narrative self. Perception is a simulation, a controlled hallucination that is continually tested and refined by the inputs coming from our eyes, ears, skin and internal receptors, as well as the emotional response our legacy brain has expressed to those stimuli. Even the experience of being you is itself a perception. All your isocortex is doing is making an educated guess at how best to represent what you are as a more or less coherent entity. You are what you model yourself to be. The you that you are being is a digital twin.
Evolution has driven that model, and the modelling apparatus, to be good not at making an accurate representation of reality but at keeping us alive. What is really out there is almost certainly nothing like the model we are seeing, hearing or feeling in our heads. Indeed, if modern theories of physical reality are to be believed and we could see it for what it actually is, we would surely go mad. What we actually perceive of the world is something more like a 3D version of the Desktop on your PC. You’re not seeing a tree or a house or mountains on the horizon, you’re seeing an iconic object of a tree, a house or mountains. Your isocortex continually proposes webs of objects and object parts that might match the signals coming in, identifying what and where they are and predicting their movements. It may propose a few hundred of these each moment and test them all against the sensory input to identify the best fit. The winner goes into the final model and is the information of which you become conscious. It is what you see, hear and feel (taste and smell are a little different as these are still processed exclusively by the older part of the brain, which is why they can make such a direct emotional connection to memories).
Every one of the elements that the isocortex puts in its model requires a web of references to other elements, and to describe the world around us those references have to include information about where one object is believed to be in relation to another. Everything we perceive exists in a 3D model. We have evolved to think spatially.
I believe that, to go further, the mammalian isocortex began to annotate or label its objects. The model gained a kind of index of terms, and those object labels themselves became separate objects in their own right, linking into the web of visual, auditory and somatic memories. Our ancestors then took that a long way further. We sent links back in the other direction, out of the internal model and its index of annotated objects, to build a two-way communication with the outside world. The brain region we call Broca’s area became dedicated to controlling the mouth so that we could convert our internal labels into something audible, while Wernicke’s area became dedicated to interpreting what came back. We had invented language.
And that opened a whole new vista of concepts for our mammalian brain to model. Suddenly we could make representations of abstract and remote concepts by describing them in terms of others and grow our complex web of dendritic connections between and around them. The model expanded hugely in reach. We can now envision things that do not and could never exist physically and connect them with things that do. We can have not only trees, rocks and rivers but also money, nations, rhyme, gods and honour.
That rich 3D model of the world has become an open platform capable of modelling almost anything. It has built in our heads a giant network of terms that are a kind of internal language built of neurons that recognise specific objects; whether those objects can exist tangibly or not is now irrelevant to their existence in the mind. We have extended that internal language by linking it to vocal, audio and now written representations, to supplement the visual and other sensory links that already defined each object. Matching the model to any of those sensory inputs will trigger not only the object associated with the best-fit neuron; it will dredge up a network of others most closely linked to it. The net has become a fishing net, and since language freed it to roam the concepts of our imagination, anything can go fishing in it. Something we think we may be seeing or hearing, or something we just happen to be thinking about. For we are no longer slaves to the present. Language freed us to be thinking beings even when little or nothing is coming in from our senses. Language freed us to remember at will. And language is impossible without that rich internal model to go fishing in.
That, I believe, is what any really valuable AI development must achieve on the route to AGI. It is a completely different approach from anything currently out there, outside a few early experiments in private research labs like Numenta. We really do have to start again. We need to think of a neuron as a simple program and set that program running concurrently billions of times in order to construct the fabric of a neural network. No more tensors or other maths shortcuts. Go back to first principles and get them right before trying to optimise the calculations. We need to design that simple program so that it will organise itself in just the way that our cortical neurons do. Model-building and prediction through competition and consensus has to be a built-in consequence of the way our programmed neurons interact. The result will be a network with a completely different topography that could really stand a chance of leading to a sentient AI.
If I’m right about this, we could still do it. But should we? Would that be entirely wise?
Something Old, Something New
Before the innovation of the isocortex, we mammals, like the reptiles, did have potent emotional states and we could react to them, as well as to complex stimuli in our immediate environment, with a certain simple logic. We were good at it, and it kept our ancestors alive for millions of generations. It still keeps our reptile relatives alive, and that legacy brain is functioning just as well in us. The isocortex may give us our self-awareness and our ability to think complex thoughts, but the old brain wasn’t switched off in mammals: it’s working just as hard, in parallel, drawing its own conclusions and competing for control of the body. And it can operate faster and get motor actions out quicker than the new brain, which has to go through it to get anything done. Sometimes, as a result, the isocortex has to explain to itself why you did what you did by modelling what appears to have happened through interpreting how your old brain felt about it. The old brain can initiate chains of action; the new one can only ask it to do so. The old brain can initiate the biochemical changes that cause emotions; the new one can only model those changes to construct higher level interpretations of them.
Without our primitive, legacy parts of the brain, there are no emotional triggers, no inciting biochemical events, no irrational responses, no impromptu aims. The isocortex is a thinking machine, not a feeling one. It is the higher brain in more ways than one. And it is this that we would be trying to mimic in order to achieve true AGI. There would be no way for such a creation, even were it to become sentient, to suffer from the poor behaviours that plague our kind. No fear, jealousy or avarice. No tribal, mob behaviour. No contempt or brutality. No desire to do anything but solve problems and no capacity to fabricate its own issues. It wouldn’t have the equipment to trigger any of those things.
General Utopia
For that reason, I think an AGI could never be anything but benign. But could it still rapidly outgrow us and leave us behind? Even if there would be a way for it to develop such an aim, or to accidentally achieve it, the idea that that could happen in minutes or hours is still, to my mind, fanciful. The iterative cycle required to continually improve one’s intellect involves learning from experience, and that can be achieved only in real time. Soaking up Wikipedia and The Pile the way that current AIs have done is not going to work when training an AGI. Only real experience of the world will do it. We’re not training a mathematical engine anymore; we are training an intelligence, and that has to place everything it learns in a frame of reference that incorporates a vast web of relationships between concepts and themes. The fact that silicon chips can conduct electricity faster than neurons is irrelevant: the bottleneck is the passage of actual events in the real world and an AGI can only experience those at the same speed we do. Real time controls access to the real world.
The Singularity that Ray Kurzweil imagined is a good story. But it is only a story; it won’t happen like that. The growth of true AGI will be far more gradual and far more aligned with our goals. We will have the time to make sure that it is.
Reading Recommendations
The line of reasoning I’ve followed in this article has been influenced by the following excellent books, though of course I don’t claim to have done any of them justice or even to agree with everything in them. But I do recommend them all!