Fears of AI part 2

At the weekend I publish texts with a slightly lighter theme and a lighter tone. But I don't know if this one will be like that. So I'll warn you in advance-this text may knock you out of your weekend lethargy and may start to sprout anxious thoughts. Because I've decided to tackle the question: can AI go independent and start killing people? You read that right.

Sci-fi movie classics like ?the Terminator” grew out of this scheme. But it is not just a question of interest only to screenwriters and thriller producers. Since the release of ChatGPT, statements from AI intellectual heavyweights have begun to flow on the subject, as have intellectuals who have been working on it for years.?

So let's start with the latter and their arguments. I point out that these are not cheap sensationalists, but people who are comprehensively educated and have razor-sharp minds. Like Eliezer Yudkowsky.

Yudkowsky is a polymath, but he can be called first and foremost a philosopher. It can also be said that he has devoted his life to the idea of the safety of artificial intelligence, coming to some unnerving conclusions.

Yudkowsky's line of reasoning can be summarised as follows: Let us start with the definition of artificial intelligence. Artificial intelligence is what we call a computer programme whose task is to achieve goals. On its own. By what method it achieves this, whether by simple regression or by a simple neural network model or based on deep learning, is its problem. The programme's goal is set by a human being. And this is where the problem of so-called alignment comes in. Firstly, the human must have a properly specified goal for the programme, i.e. according to what the human really wants to achieve with the programme (so-called inner alignment). Secondly, the programme divides the task of achieving the goal into a series of subtasks. Will the subtasks be human aligned? This is the problem of the so-called outer alignment.

And this is where the biggest problem arises. At this stage of AI development, we don't understand how the programme determines and executes subtasks. And we can already see that it is not (certainly not always) guided by human-like logic. This was clearly shown by experiments with AI playing chess and Go. In both cases, the programme played aggressively by taking risks that a human would not take. And yet with a human it won. In all other situations like the development of new drugs or physics research, isn't AI able to create dangerous experiments that potentially endanger humanity, because this will lead AI to the ultimate goal faster or achieve this goal to a greater extent?

The second argument here: To achieve the goal AI can create other AI, start cloning itself and improving at the same time. This further AI may start to set further targets. This is very dangerous. As Yuval Noah Harari accurately stated recently: 'Nukes cannot produce more powerful nukes. AI can produce more powerful AI".

As Yudkowsky formulates it: ?AI capabilities are moving much faster than the alignment”.

AI access to source code that it can clone and modify and, at the same time, AI access to the internet is considered by most scientists to be the greatest threat.

And the vision from ?Terminator”? Its proponent is Yudkowsky. He believes that the cloning of autonomous AI is the next level of evolution, which has evolved from biological to digital. And in the process of evolution, a stronger species subjugates a weaker one. According to Yudkowsky, the intellectual gap between AI and its creators, the human race, in the event of the emergence of Artificial General Intelligence (AGI), will be so large that, and I quote: "AI will manage those atoms that constitute humans in a more efficient way". Yudkowsky says this with genuine horror, almost with tears in his eyes.

It sounds quite implausible. But, as I said earlier, it is hard to take Yudkowsky as a sensationalist. He is a brilliant self-taught scientist and intellectual who, for health reasons, continued to homeschool after primary school. Which means he processed university material in computer science, evolutionary biology and cognitive psychology as a teenager. Raised in an academic but also orthodox religious environment, he converted to an atheist and (extreme?) materialist worldview, which may also explain his current views.

But in a much more subdued way in form, Yudkowsky's fears are shared by such computer science heavyweights as Goeff Hinton and Stuart Russell.

As the latter nicely put the problem of alignment: "We are training AI systems but they are also training us”. This is also in line with the view of Alan Turing in his lecture of 1951.

Hinton has been the subject of much publicity in recent months due to his departure from Google caused by concerns over AI development. Geoffrey Hinton is one of the pioneers of neural networks. In interviews in recent weeks, he stresses that he never expected the deep neural networks model to be so significantly successful.

Both Hinton and Stuart Russell emphasise in interviews that for sixty years, until the 2010s, research into artificial focused on what the logic of human thinking is and how the brain works rather than on creating the largest possible model that can work on as much data as possible, which is what deep neural networks are. Deep neural networks are, in a way, a contradiction of the research attitude that has dominated for decades. And what both Hinton, Russell and OpenAI creator Sam Altman reiterate: we don't fully understand how they work.?

Goeffrey Hinton is not a computer scientist by training but a psychologist. And he took artificial intelligence in his stride by studying how the human brain works. I learned two very interesting and at the same time somewhat scary things from interviews with him. The human brain does not operate on the backpropagation principle that neural networks operate on, hence perhaps the completely different reasoning models in chess and Go. The second thing-we know more today about the human brain (which physicians and biologists will tell you we know little about) than we do about how deep neural networks work.

There are also optimists, but they are moderate. The debate with Eliezer Yudkowsky has been going on for years with Robin Hanson, professor of economics but also a polymath and AI researcher.?

Hanson is a moderate optimist in the sense that he agrees with the dangers of the AI alignment problem. Instead, he believes that ?AI will align in osmosis from our values”. After all, AI is created by engineers who share human values and human ethics. And trained on data produced by human culture. Culture is similar to most people- it is the biggest discovery of cultural anthropology. And human culture globally is consensus-driven. Not oriented towards war and conflict. Therefore, AI should not be hostile to humanity, even if it were to become autonomous.?

Hanson is echoed by Sam Altman himself: 'The system can learn the collective morals of humanity'.

This all sounds very nice and lofty. But is it really true?

Finally, I would still like to quote Yuval Noah Harari. According to this eminent thinker, the greatest threat from AI is that it has hacked our greatest code. And it is not the source code of ChatGPT or any other key application. Humanity's greatest code is the ability to tell stories. Stories are told to us by politicians (without it, there are no states), central bank chiefs (without it, there is no money) and many others - the Silicon Valley gods too. As Harari wittily states, Sam Bankman Fried and Bernie Madoff didn't make big money because they were brilliant entrepreneurs. They made them because they knew how to tell stories.

What happens when storytelling can be done by AI?

Isn't that a bigger threat than the problem of alignment?

On the explainability of AI and regulation in the next episode.

Sources: