【Opinion】 DeepSeek, OpenAI, and the Race to Human Extinction

By Stuart Russell

The shock waves continue to reverberate after the arrival of a new AI system from a small Chinese AI company called DeepSeek. Stocks of major AI-related companies, including Nvidia, lost over $1 trillion in a single day. Why?

The early reports suggest that DeepSeek is similar in design to some of the recent AI "reasoning" systems such as OpenAI's o1, in that it combines a large language model with the capability for executing multiple steps of reasoning, looking for a way to solve a problem or answer a complex question.

It is claimed that DeepSeek is roughly as good as the latest systems from U.S. companies, but it's probably too early to say.

Chatbot performance is a complex topic. For example, it might be impressive if a system scores well on Math Olympiad tests, but less so if it's been trained on thousands of questions from exactly those tests.

And perhaps not surprisingly, OpenAI claims that DeepSeek has been cheating by accessing o1 to train its models. Some results I've seen also suggest that DeepSeek fares far worse than o1 in "red-teaming" tests that measure a system's willingness to behave badly.

If the claims of DeepSeek's excellent performance hold up, this would be another example of Chinese developers managing to roughly replicate U.S. systems a few months after their release.

The general outlines of how OpenAI's o1 works have been known for quite a while—even before it was released—so it's not all that surprising that it can be roughly replicated.

What's surprising is the claim that the total training cost was only $6 million and it was done using "only" a few thousand GPU chips. (Reports vary widely on exactly what was used.)

Both of these numbers have caused grief in the markets because U.S. companies such as Microsoft, Meta, and OpenAI are making huge investments in chips and data centers on the assumption that they will be needed for training and operating these new kinds of systems. (And by "huge" I mean really, really huge—possibly the biggest capital investments the human race has ever undertaken.)

If that assumption is false and it can be done much more cheaply, then those investments are mostly a waste of money, and future demand for Nvidia's chips in particular will be much lower than predicted.

To be honest, the race to build larger and larger data centers had already started to look like the race between the U.S. and the Soviet Union in the 1960s to build and test larger and larger bombs: They got as far as 50 megatons before realizing it was all rather pointless, but in the process they wasted billions of dollars and dumped enough radioactivity into the atmosphere to kill 100,000 people.

The "AGI race" between companies and between nations is somewhat similar, except worse: Even the CEOs who are engaging in the race have stated that whoever wins has a significant probability of causing human extinction in the process, because we have no idea how to control systems more intelligent than ourselves.

In other words, the AGI race is a race towards the edge of a cliff.

Stuart Russell is distinguished professor of computer science, University of California, Berkeley, and Smith-Zadeh professor in engineering; professor of cognitive science; professor of computational precision health, UCSF; and honorary fellow of Wadham College, Oxford.