The computing power game behind ChatGPT Chinese companies urgently need to break the shackles of the United States

Huaxi Securities predicts that the global AI software market will reach US$126 billion in 2025, with a compound growth rate of 41.02% from 2021 to 2025.

Behind the prosperity of ChatGPT is the support of astronomical computing power.

According to estimates, in terms of AI training servers, a single large language model training drives the demand for AI training servers to about 200 million U.S. dollars; in terms of AI inference servers, such as ChatGPT, it can drive the demand for inference servers to about 4.5 billion U.S. dollars in the early stage.

For a time, ChatGPTs emerged one after another, and the AI server track behind them also began to rise accordingly.

Computing power determines ChatGPT

Computing power is the core engine of the large model, and its calculation formula is very simple: how many GPU chips can generate as much computing power, and the number of high-end GPU chips can directly affect the computing power.

**The computing power required by ChatGPT is not fixed, but increases step by step. The smarter ChatGPT is, the price behind it is that more and more computing power is required. **

According to media speculation, the GPT-3 training cost is expected to be 5 million US dollars per time, the GPT-3 model needs to cost about 1.4 million US dollars in training costs, and Google's PaLM model needs to cost about 11.2 million US dollars in training costs.

According to Microsoft executives, the AI supercomputer that provides computing power for ChatGPT is a large top-notch supercomputer built by Microsoft with an investment of US$1 billion in 2019. It is equipped with tens of thousands of Nvidia A100 GPUs and more than 60 In total, hundreds of thousands of Nvidia GPUs are deployed in the data center.

In order to meet the ever-increasing demand for ChatGPT computing power, Microsoft announced that it will launch a series of Azure scalable AI virtual machines based on Nvidia's latest flagship chip H100 GPU and Nvidia's Quantum-2 InfiniBand network interconnection technology to significantly accelerate the development of AI models .

It seems that behind ChatGPT is full of Nvidia, Nvidia and Nvidia.

In fact, Nvidia, as the hardware overlord, not only occupies most of the market in the consumer market, but also is the number one choice in the field of AI server chips.

Rare things are more expensive. At present, Nvidia's flagship chip H100 has increased in price by nearly 70,000 yuan in a week, and the price is generally as high as 300,000 yuan; Yuan rose all the way to 90,000 yuan, an increase of more than 50%.

Not only can it not be bought at higher prices, but even the United States has banned Nvidia from selling chips. In August last year, the U.S. government issued an export control policy, prohibiting Nvidia from selling the A100 and H100 chips to China.

In order not to lose the Chinese market and comply with US export controls, Nvidia subsequently launched performance-castrated versions of the A800 and H800 chips. But these two chips were also snatched up by the market in short supply, and the price has also risen accordingly.

Led by Baidu, Ali and Tencent, most Internet companies in China have announced their entry into the large-scale model. According to market statistics, since ChatGPT, the number of large models to be launched in China this year has exceeded 10.

If you want to reach the level of ChatGPT, you need at least 3,000 A100 chips, which is 270 million RMB at a price of 90,000 per piece to complete the deployment of a large model; 10 large models need 30,000 A100 chips, 2.7 billion RMB .

In addition to the cost of later training, the required chips are even more astronomical. But judging from the current delivery time of Nvidia, it is not easy to buy enough chips.

In a trance, the mining card era came again.

Nvidia sitting in the air again

When the virtual currency was hot in the past few years, as a graphics card provider necessary for mining, Nvidia made a huge profit of 4.8 billion U.S. dollars in a few years. Now relying on ChatGPT to live a second life, let history repeat itself again.

Facing the surge in market demand, Nvidia, which took advantage of the wave of AI to turn around, launched a computing power leasing service.

On March 21, at the 2023 GTC conference, NVIDIA founder and CEO Jensen Huang launched the NVIDIA DGX Cloud, which can provide enterprises with the infrastructure and software needed to train advanced AI models. Each instance of DGX Cloud is equipped with 8 H100 or A100 80GB GPUs. Enterprises can rent DGX Cloud clusters on a monthly basis in the form of cloud leasing. The price starts at $37,000 per instance per month.

**Is there really no substitute for Nvidia? Why do companies prefer to choose leasing rather than other GPU chip manufacturers? **

According to IDC data, domestic GPU servers will account for more than 88.4% of the domestic server market in 2021, and products using NVIDIA will account for more than 80%.

**The chip required by the AI large model has higher requirements on the precision of processing information and the speed of computing power. In the field of supercomputing, the double-precision floating-point computing capability FP64 is a rigid indicator for high computing power calculations. Nvidia's H100 and A100 are currently the only chips with these capabilities. **

The sales of Nvidia chips are not the only thing that is stuck in the United States. Technology, equipment, and materials all restrict the research and development of Chinese companies. However, under the heavy restrictions of the United States, Chinese companies still ran out of several dark horses under pressure.

According to the latest "China Accelerated Computing Market (Second Half of 2021) Tracking Report" released by IDC, the scale of China's AI server market will reach 35.03 billion yuan in 2021, a year-on-year increase of 68.6%.

In the field of enterprise-level GPU chips, Chinese manufacturer Biren Technology will launch the BR100 chip in 2022, Tianshu Zhixin will launch the Zhikai 100 chip, and Cambrian will launch the Siyuan 270 chip.

Among them, Biren Technology said that the BR100 has the highest computing power in the world, and its peak computing power has reached more than three times that of the flagship products on the market. The computing power reaches the PFLOPS level.

Although the data is good, it lacks the crucial ability to process FP64, and it still cannot completely replace the two brothers Nvidia H100 and A100.

Moreover, the CUDA platform used by Nvidia has already become the most widely used AI development ecosystem. It only supports Nvidia’s Tesla-based GPU, which cannot be replaced by domestic chips at this stage.

Although Chinese chip manufacturers are catching up in the field of GPU chips, the technological gap and the bottleneck in the United States are still key issues, and it will take some time to work hard.

More than AI server

Not only AI servers and GPU chips, but also the storage market are rising all the way with the help of large-scale models.

**ChatGPT's operating conditions include training data, model algorithms, and high computing power. The underlying infrastructure with high computing power is the basis for massive data and training. **

The most obvious feature is that after several iterations of ChatGPT, the number of parameters has increased from 117 million to 175 billion, an increase of nearly two thousand times, which also brings great challenges to computing and storage.

**As the new era of AI begins, it is expected that the amount of global data generation, storage, and processing will increase exponentially, and memory will benefit significantly. Computational storage is an important cornerstone of ChatGPT. With the entry of technology giants such as Alibaba and Baidu into ChatGPT projects, the overall computing storage market demand will further increase rapidly. **

As the AIGC continues to flourish, Beijing, Shanghai, Guangzhou and other digital economy-developed regions have also introduced policies to promote the construction of intelligent computing centers. For example, Beijing proposes to build a new batch of computational data centers and artificial intelligence computing power centers, and cultivate them into artificial intelligence computing power hubs by 2023; Shanghai proposes to build a batch of high-performance, high-throughput artificial intelligence computing power centers , Promote the construction of public computing power service platforms, etc.

And all walks of life will face the baptism of ChatGPT. Under the new wave of artificial intelligence, AI-related industries will usher in a broad market space.

And Chinese companies are bound to break through the constraints of the United States and break the shackles of unfairness.