Big AI models spark arithmetic gap as China's four biggest internet giants place $5bn order for Nvidia chips

In June this year, the media said, byte jumping this year to nvidia ordered more than 1 billion dollars of GPUs, byte arrived and did not arrive A100 and H800 total 100,000, only byte a company this year's order may be close to nvidia last year's sales in China's commercial GPUs and total.

At present, AI large model of the huge arithmetic demand, is leading to GPU processor "a core hard to find". Recently, according to the financial times, citing two unnamed people close to nvidia reported that baidu, byte jump, tencent and alibaba to nvidia order 5 billion dollars of chips.

Among them, NVIDIA will ship a combined total of about 100,000 A800 chips worth $1 billion this year, and another $4 billion worth of chips will all be delivered in 2024. However, NVIDIA and the above four domestic companies have not commented on the above news.

AI arithmetic demand is strong

GPU is able to process large amounts of data quickly and accurately due to its advantages of graphic rendering and parallel computing of massive data, and its market value is gradually becoming more prominent, and it is widely used in AI, image rendering, scientific computing and other fields. Among them, AI, autonomous driving and gaming markets are the main scenarios for GPU demand growth.

At the end of 2022, ChatGPT came out and quickly set off the AI trend around the world. NVIDIA, which has been betting on the future of AI, has become the company with the largest gain in the AI big model trend, and its market value has reached a trillion-dollar market value.

In March, NVIDIA re-released the H100 NVLINK chip at the GTC conference, a GPU designed specifically for Large Language Modelling (LLM) with a Transformer acceleration solution for processing ChatGPT. Compared to NVIDIA's HGX A100 GPU chip, a server equipped with four pairs of H100 and dual GPU NVLINK Compared to NVIDIA's HGX A100 GPU chips, a server with four pairs of H100s and dual GPU NVLINKs is up to 10 times faster, reducing LLM processing costs by an order of magnitude.

Currently, NVIDIA's technological leadership in high-performance GPUs, with its A100 and H100 AI chips, is the core power of large language models like ChatGPT. Some cloud computing professionals have described 10,000 NVIDIA A100 chips as the arithmetic threshold for a good AI model. And the AI supercomputer that Microsoft built for OpenAI to train its models is equipped with 10,000 NVIDIA GPU chips.

Recently, NVIDIA launched the next-generation version of its GH200 Grace Hopper superchip. The chip will be the world's first GPU chip equipped with HBM3e (High Bandwidth Memory 3e) memory. Compared to the current generation, the latest version of the GH200 superchip has a 3.5x increase in memory capacity and a 3x increase in bandwidth; compared to the hottest H100 chip, it has a 1.7x increase in memory and a 1.5x increase in transfer bandwidth.

According to GlobalMarketInsights, the global GPU market is expected to continue to grow at a CAGR of 25.9 per cent to reach a size of $400 billion by 2030. Among them, the continued introduction of large language models in the field of AI and the growing number of participants is expected to drive rapid growth in demand for GPUs at the model training end and reasoning end.

NVIDIA said in a statement, "Consumer internet companies and cloud providers invest billions of dollars annually in data centre components, often placing orders months in advance." NVIDIA CFO Kress also said that the current demand in the AI computing power market has exceeded the company's expectations for the coming quarters, and there are already more orders than can be made.

GPU prices are rising again and again

In 2022, the United States increased the AI chip export restrictions to China. In order to circumvent the restrictions of the policy, NVIDIA to the Chinese market exclusively for the A800 processor to replace the A100 GPU, however, in the AI large model computing power demand, even the performance is limited to the "exclusive version" of the GPU is also in short supply.

In the past few months, driven by the demand for AI large model computing power, NVIDIA's A100, H100, A800 and H800 and other high-performance GPU chip prices continue to rise. In particular, the flagship chip H100, in mid-April in the overseas e-commerce platform has been speculated to more than 40,000 U.S. dollars, and there are even sellers priced at 65,000 U.S. dollars. At the same time, NVIDIA's China special edition A800 and H800 chips have also been coaxed.

It is worth mentioning that previously, a number of media reports, the U.S. government is brewing the implementation of new chip export restrictions, mainly related to NVIDIA's AI chips exported to China. Although the new restrictions have not landed, but the market has long been "windy", the price of GPU has risen again and again.

In early July, according to surging news reports, nvidia in China, an agent said, now nvidia A800 chip a day a price, if the purchase volume can be more than 100,000 yuan / piece, less if there is no 110,000 yuan / piece can not get, "taking into account the rumours of a ban, we are more reluctant to sell, more than a week since the rate of increase of 20% to 30%.

At the end of July, Tencent Technology's report also mentioned that NVIDIA chips are being looted, said in the chip distribution circle, NVIDIA's China special version of the A800 and H800 price from the original 120,000 yuan or so, into the current 250,000 or even 300,000 yuan, and even up to 500,000 yuan a piece.

In June this year, there are media said, byte jumping this year to nvidia ordered more than 1 billion dollars of GPU, byte arrived and did not arrive A100 and H800 total 100,000 pieces, only byte a company this year's order may have been close to nvidia last year's sales of commercial GPUs in China and the total. Sources familiar with the matter said that byte jump has stockpiled at least 10,000 NVIDIA GPUs. byte jump also ordered nearly 70,000 A800 chips, to be delivered next year, worth about $700 million.

NVIDIA's chief financial officer said in June that restrictions on AI chip exports to China "will lead to a permanent loss of opportunity for the U.S. industry," and the company does not expect to be materially affected in the short term.

At present, nvidia order visibility has been to 2024, high-end chip is very tight. With the current scheduling progress, even the A800/H800 to the end of this year or next year before delivery.