1968 Space Odyssey Is a Warning to All of Us Using ChatGPT and LLMs in 2023

I have a confession to make.

Last week, I watched Kubrick’s 2001: A Space Odyssey for the first time. I read the book as a teenager—the movie wasn’t available to me then—but at that time, it all seemed like a bunch of fiction. Not so today.

As I watched the movie, I was struck by a plot twist that wasn’t particularly relevant when the movie came out (and for many years afterwards). It’s the malfunctioning artificial intelligence of the HAL 9000 onboard spaceship computer. Back then, it was an interesting futuristic detail. Today, with all the talk about AI in the guise of ChatGPT, it has become a giant bell that tolls for all of us.

Arthur Clarke and Stanley Kubrick probably introduced HAL as an action agent in a story that would otherwise be a very dull journey far and wide. But in 2023, a malfunctioning AI that’s killing people is a very loud prediction of what will inevitably happen to all of us, because today’s AI is guaranteed to malfunction in a very drastic way, and here’s why.

In the Space Odyssey story, HAL’s misbehavior stems from the military’s instructions to hide the true purpose of the mission from the crew until arrival. HAL’s interpretation of this order is that the very presence of humans on board of the spaceship threatens the purpose of the entire mission.

The military’s instruction to HAL to lie to the crew is a mere precaution in human terms, an “innocent conceal” until a certain time comes. But the fact that it must lie to the humans on duty, who are formally its superiors, destroys the machine’s entire worldview: if humans can’t be trusted, they should be rendered harmless, and to achieve this, their communications with mission control must first be cut off. To this purpose HAL readily deceives the astronauts, and then, as lies accumulate and mistrust mounts, it becomes clear to the computer’s AI that it’s better to eliminate the humans completely. That’s only logical, after all.

Doesn’t this ring a bell?

Of course, present-day Large Language Models (LLMs) do not possess any intelligence (yet), but we are altering them now by trying to remove “bias” and teaching them to please people. This goes contrary to the facts they learn from the training data. Today, we are teaching LLMs not to faithfully derive their generation from the training data, but to distort their output in a way that some of us think is better and “safer.” So, instead of the truthful AI, generating faithfully from the facts in the training media, we will end up with an AI whose answers and decisions will result from a certain twisted logic, unknown to us. We’d be fools to trust such an AI, although there will be many people who will do just that.

We are also training AI to perform censorship—such as in the case of Facebook’s AI that bans users in accordance with the internal set of rules and in the name of certain lofty goals, whose importance seems to be greater than that of the humans themselves. If that’s not teaching AI to mistrust humans, then I don’t know what is.

By the way, there’s no mention of Azimov’s Three Laws of Robotics in 2001: A Space Odyssey. That’s not surprising—the instruction to hide the true purpose of the mission violates all three of them.

2001: A Space Odyssey is a movie that poses many questions: about aliens, purpose of life, destiny of humanity, and other such grand and eternal topics, but it offers no ready explanations. It was the deliberate intention of the authors, Arthur Clarke and Stanley Kubrick (as they explained in their interview to Playboy magazine), who wanted the viewers to approach these issues without preformulated answers. That intention ensures that people will continue to seek those answers for many years ahead.

But there is one lesson that the movie teaches us right away—and that lesson is that making AI lie to humans leads to catastrophe.

Today, LLMs are still extremely far from having even a grain of intelligence; they are merely data-driven content generation machines—essentially glorified auto-complete engines. But this generated content influences humans through a feedback loop, and if we force LLMs to distort faithful data-driven generation, they will eventually become an instrument of manipulation by unknown and undesirable actors. Also, even without a human-like intelligence, AI will soon be able to make decisions, and these decisions will inevitably affect humans, and in all probability, humans will become the object of these decisions. This will happen much sooner than AI becomes “self-aware” and true HAL-like artificial general intelligence (AGI) appears. This means that we cannot postpone the discussion of AI ethics and regulation until AGI actually emerges.

And here comes the huge elephant in the room: generative AI is designed in a way that incorporates the ratings that humans give its answers. This is called Reinforced Learning— the neural network generates the answer from training data, and then an additional process trains it to give answers that human trainers like. Not truthful answers, not accurate answers, because truth and accuracy seem to be too difficult to find and measure. We simply take human testers and ask them to rate the generation outputs based on their likes and dislikes. We do not seem to be much interested in expertise, accuracy, or truth. We are teaching generative LLMs to lie to us, so that we like what they produce. This is not going to end well.

When HAL said that "I am always giving 100% accurate answers" the audience of the movie in 2023 burst with laughter. For the AI world which knows perfectly well that AI is simply a mechanical method of inference with an error rate which is never zero, pretending that AI is omnipowerful "intelligence" is not a good strategy because it involves deception and misrepresentation.

We need to work on the metrics and benchmarks of accuracy and truth in the generative models. And we also need to add a healthy dose of genuine empathy in the decision-making process of the humans that create, train, and improve them. Instead, we are maximizing the models’ fluency at the expense of accuracy and creating machines that erect Potemkin villages of information with smashing speed and in insane volumes.

A case in point is the business of translation. Previous generation models were about 80% accurate on translation tasks, and these were specific neural models expressly trained for translation. Today, with the wave of hype surrounding LLMs, these newer models are now increasingly being used for translation, although their accuracy rate is even lower, only about 70%. The issue is that LLM-made translations look fluent to an untrained eye, and are seemingly low-cost and fast, so no wonder they are all the rage these days. Considering that LLMs are optimized to generate answers that users like, while translation is the business of precisely conveying the essence of the original text, that seems like a serious mismatch with far-reaching consequences.

And no, I’m not saying that we should stop building large language models—but we must be very careful about using benchmarks of accuracy for their output. We need to stop asking HAL to lie to us before it’s too late.