AGI & The Vulnerable World Hypothesis (part 1): GPT-4 Doesn’t Understand

"The first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control."

― Nick Bostrom, Superintelligence: Paths, Dangers, Strategies

Backstory: GPT-4 Doesn’t Understand

Last summer, I posted a video discussing the negative consequences that an undeveloped technology created with good intent can have. In the video, I used the example of an application that would track bears in Alaska. However, the real catalyst was GPT-3 in addition to a demo series my team designed to show the positive and negative applications of this rapidly expanding technology. Ultimately, we scrapped the series when we realized it was turning into the RPA + GPT equivalent of the Anarchists Cookbook.

I closed the Alaska Bear video by citing a future part 2 video on the Vulnerable World Hypothesis, and how companies may want to apply the preventative measures it introduces. The script has been gathering dust for the better part of a year.

Triggered by the Future of Life Institute’s March 22nd “Open Letter” calling for a six-month moratorium on training “AI systems more powerful than GPT-4”, I began a rewrite of that script reflecting on GPT. Last week, US lawmakers fueled the fire with the “Block Nuclear Launch by Autonomous AI Act,” an unnecessary waste of taxpayer money to reinforce something already banned. After writing and rewriting what seems like endless spin-offs of the same topic, it seems time to put down the pen and post something.

In this Part 1 post, I hope to re-introduce some reality around GPT, Artificial General Intelligence, and the direction of these developments. While this report is not exhaustive, I have included links to further readings.?

Part 2 will cover the Vulnerable World Hypothesis as a framework for guardrails around Artificial General Intelligence. Part 2 has not yet been posted.

Future of Life Institute – Selling Tickets to the Hype Train

In 2015, Nick Bostrom, whose work is the topic of this article, joined Stephen Hawking and 8,000 other signatories in the Future of Life Institute’s (FLI) Open Letter: Research Priorities for Robust and Beneficial Artificial Intelligence. Unlike the initial release of the Future of Life Institute’s (FLI) March 2023 Open Letter, the 2015 letter included a prescription to reduce risk while maximizing the social benefit of AI.

In addition to lacking a prescription, in 2023 the Future of Life Institute’s letter pushed a fearmongering narrative that benefits supporters such as Elon Musk. FLI president Max Tegmark dismissed the intention to hinder OpenAI’s corporate advantage. However, just a few weeks after its release, Musk created X.AI, which he publicly positioned to rival OpenAI (see "preference modification" in part 2).

While FLI has stirred interest by pushing fears of an AI apocalypse, they have also drawn criticism from industry leaders. Co-Authors of the 2021 peer-reviewed paper On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?" responded with a statement to FTI’s “AI Pause” Letter.

“It is indeed time to act: but the focus of our concern should not be imaginary "powerful digital minds." Instead, we should focus on the very real and very present exploitative practices of the companies claiming to build them, who are rapidly centralizing power and increasing social inequities.”

― Timnit Gebru (DAIR), Emily M. Bender (University of Washington), Angelina McMillan-Major (University of Washington), Margaret Mitchell (Hugging Face)

“Stochastic Parrots” is the first supporting reference under Future of Life Institute’s letter, followed by Nick Bostrom’s book “Superintelligence.” However, Bostrom and other 2015 signatories, including Jen-Hsun Huang?(NVIDIA CEO), the co-founders of DeepMind, J. B. Straubel (former Tesla CTO), and Thomas Mueller (SpaceX co-founder), did not sign FLI’s latest open letter.

Unusual Benchmarks

As mentioned, the Future of Life Institute’s 2023 Open Letter petitions to “immediately pause for at least six months the training of AI systems more powerful than GPT-4”. However, this is an unusual barrier. GPT-4 is a Large Language Model (LLM). Broadly, language models are trained to predict the most likely next word in a sentence based on the previous entry. An advanced type-ahead capability is not the fear-inducing theatrical depiction most of the public pictures when reading about a world-ending Artificial Intelligence.

If movies have strictly shaped your narrative on AI, Nick Bostrom’s 2016 book “Superintelligence: Paths, Dangers, Strategies,” is a worthwhile read or listen.

“Granted, there is still that picture of the Terminator jeering over practically every journalistic attempt to engage with the subject.”

― Nick Bostrom, Superintelligence: Paths, Dangers, Strategies

When reflecting on limiting AI by GPT -4’s capabilities, a few questions must be considered:

• How does a Language Model serve as a benchmark for broader Artificial Intelligence?
• Is the ability to memorize answers to multiple-choice questions applicable to all AI?
• Do we use an average of GPT-4’s benchmarks against MMLU, HellaSwag, ARC, and GSM-8K as the barrier, or pick one?
• Do we cap precision, recall, accuracy, or F1 score? (see Accuracy Doesn’t Matter)

We could cap the training parameters. News outlets report that GPT-4 has either 1, 17, 100, or 175 trillion parameters, but OpenAI will not release the parameter count.

Finally, despite the Future of Life Institute targeting GPT-4, the letter’s FAQ is focused on Artificial General Intelligence (AGI). This is an entirely different spin than the content of the “AI Pause” letter, which prompts three critical clarifications:

• GPT-4 is not AGI
• GPT-4 is not a direct path to AGI (by itself)
• GPT-4 is not a development ceiling relevant to AGI

GPT-4 is not the road to Artificial General Intelligence

Artificial General Intelligence (AGI) refers to an AI system that can understand, learn, and apply knowledge across a wide range of tasks and domains, essentially matching or surpassing human intelligence. However, GPT and its LLM cousins cannot transfer learning to previously unseen situations or recognize their limitations. The generated text has no basis in intent, the reader’s state of mind, or a contextual model of the world. In fact, without a complete redesign and retraining, GPT could not begin to attempt to understand the meaning of words.

The Hugging Face course on Transformer Models is helpful to learn why auto-regressive decoder-only models like GPT are similar to fancy auto-fill. While GPT is a decoder-only model, I would recommend watching the encoder-only video first for important context.

Encoder Models - https://huggingface.co/course/chapter1/5?fw=pt

Decoder Models - https://huggingface.co/course/chapter1/6?fw=pt

GPT4 has shown that LLMs can be highly beneficial research assistants, creative prompts accelerators, and type-ahead code generators. By the time GPT -5 is released, we may need to rethink the 72-year-old concept of a Turning Test as a qualifier for artificial intelligence. My current favorite alternative is the Wozniak “Coffee Test.”

A machine is required to enter an average American home and figure out how to make coffee: find the coffee machine, find the coffee, add water, find a mug, and brew the coffee by pushing the proper buttons.

In reality, an AGI will likely require many complementary technologies to operate in tandem, like a highly advanced "Hyper Automation.” Consequently, LLMs will probably help an AGI become a better conversationalist. However, auto-regressive decoder-only models alone are not a direct path to AGI.

NOTE: Advancements such as AutoGPT, and the integrations on HuggingFace, progress the path towards the possibility of an AGI. However, pairing automation tools and code with an NLP/NLU/NLG is not new or limited by capping LLM development at GPT-4.

Can we stop Artificial General Intelligence?

Nick Bostrom’s book “Superintelligence” contains a survey of expert communities that project a 50% median estimate of human-level machine intelligence (HLMI) by 2040 and 90% by 2075. However, every AI Hype cycle for 50 years has been broken by an AI Winter due to oversold promises leading to disappointment and funding cuts. OpenAI holds promise to break this through “self-funded” commercial offerings like GPT and DALL-E.

Pausing development provides little barrier to the Skynet scenario the general public is concerned about. Even if GPT were the route to AGI, one research team that doesn’t play by the rules would negate all benefits of a pause. Nick Bostrom’s paper, Racing to the Precipice: a Model of Artificial Intelligence Development, provides a quantitative explanation of the problem.

Advocates for the Future of Life Institute must know that competing companies and nations will not halt for six months (technology relinquishment/preference modification), especially considering there are no current enforcement mechanisms (preventative policing/global governance). Consequently, FTI’s proposed moratorium, based on “Asilomar AI Principles,” which were written by FLI, is a humorously opportunistic reach for attention, a transparent self-serving award for donors, nonsensical, and unactionable.

Superintelligence is a challenge for which we are not ready now and will not be ready for a long time. We have little idea when the detonation will occur, though if we hold the device to our ear we can hear a faint ticking sound.”

― Nick Bostrom, Superintelligence: Paths, Dangers, Strategies

Meanwhile, I agree that world leaders need to make significant advancements in preventative measures related to creating an Artificial General Intelligence, which could unintentionally spiral (Type-0 example). The Vulnerable World Hypothesis offers guidance we can weigh into future policy. To learn about Vulnerable World Hypothesis, I encourage you to either read the original paper at Vulnerable World Hypothesis or continue to part 2 of this post for an abbreviated explanation with many historical examples.?Part 2 has not yet been posted.