THE SINGULARITY AND UNINTENDED CONSEQUENCES OF AI

One of the main protagonists in the book THE GOD VIRUS discusses the idea of a singularity and whether it is a flawed concept.

“Once an AI reached the singularity, the classic idea was it would create an even smarter AI and that would create a smarter one and so on. That concept was unlikely. Once an intelligence reaches a certain level of awareness, it also reaches an idea of self. Why create the thing that will replace you on purpose? No, more likely that the AI self improves, updating subsystems with full regression testing to ensure it hasn’t changed who it was.”

This doesn’t mean it won’t get smarter and more capable, but this isn’t tied to the singularity. And it isn’t the exponential loop some predict.

What will the unintended consequences be of creating intelligence? Even the current LLMs built on the transformer architecture was born from one such unintended consequence. The transformer architecture was originally developed to improve machine translation, and it was a surprise how well it performed in other areas.

I don’t think we need self-aware machines, but I wonder if such a separation is even possible once we’ve reached a certain level of complexity.

And what is to say that the instructions we give the model might not create this agency. When Asimov defined his robotic laws he ended up with a zeroth law: “A robot may not injure humanity or, through inaction, allow humanity to come to harm.”

It is very easy to see how such a law could have very harmful consequences. An AI provided with such wide-ranging instructions that are open for interpretation could take actions that would harm or kill human beings.

What do you think unintended consequences could be?