It should be quite rare in any field for an originally highly mature market to gradually blossom from a duopoly. The cover story of this issue starts from the dynamics of some new GPU companies and discusses the current market status and future trends of GPUs in data centers today.
In 1993, when Nvidia was first established, the market for PC and professional graphics accelerator chips was booming. But after 10 years (2003), only Nvidia and ATI (later AMD) survived in this market.
Since then, the application scenarios of GPU have also moved from simple game graphics rendering to enterprise HPC, supercomputing, data center acceleration, and artificial intelligence (AI) applications. In the past two years, "International Electronic Business" has been paying attention to Nvidia's financial report. Since the middle of last year, the revenue of Nvidia's data center business has officially surpassed that of the game business-the data center has since become Nvidia's most profitable business unit.
It is worth mentioning that, as consumer electronics has entered a downturn period, Nvidia's FY23 Q3 (the third quarter of fiscal year 2023, as of October 30, 2022) game business revenue has fallen sharply, with a year-on-year decline of more than 50%, but the data The core business remains strong -- revenue was $3.833 billion, up 31% year-over-year. Data center business revenue accounted for 64.6% of Nvidia's total revenue for the quarter.
Since the beginning of this century, the global GPU market has formed a duopoly. In the following ten years, GPUs and graphics cards were dominated by Nvidia and AMD. But in the past two years, in the global GPU market, especially the Chinese market, many companies have entered this field. At the end of last year, statistics from market research firm Jon Peddie Research showed that a total of 18 companies worldwide were developing or manufacturing GPUs. Of course, this data also includes the mobile market and upstream GPU IP companies. However, even if these companies are not counted, there are still 11 players from the PC and data center market, including Intel, Biren, Hanbo, Tianshu Zhixin, Innosilicon, Mu Xi, Moore Thread and so on.
It should be quite rare in any field for an originally highly mature market to gradually blossom from a duopoly. The cover story of this issue starts from the dynamics of some new GPU companies and discusses the current market status and future trends of GPUs in data centers today.
The "new life" and "plunder" of GPU
In the past two years, in the report of the "GTC Nvidia Developers Conference", "International Electronic Business" mentioned: "For Nvidia, these years are indeed a 'golden age'-starting from 2015, Nvidia's business Both revenue and net profit have soared, and these two values even once increased to triple digits-Nvidia’s performance makes it look like a new company rather than a mature company.”
The blowout growth of Nvidia's performance is inseparable from the expansion of new businesses. Observing the past stock price changes of Nvidia, we can see that from 1999 to 2015, the company's stock price has been tepid. By 2015, its stock price was only 4-5 yuan. But in 2021, the company's stock price once soared above $300 (see Figure 1 for details). What happened to the global GPU market during 2015-2021?
Figure 1: Nvidia’s share price has doubled by dozens of times
Since the 2016 fiscal year, the word "Datacenter" has appeared frequently in Nvidia's annual report. In the past, this business has been called "Enterprise (enterprise)" business. Although Nvidia's data center business revenue now exceeds 60% of the company's total revenue, the rapid increase in the value of this business is also related to the downturn in the consumer electronics market. On the whole, the upward trend of the data center business has lasted for dozens of quarters.
Today, data centers and enterprise GPUs are Nvidia's biggest revenue generators. According to IDC data, by 2021, Nvidia will occupy 91.4% of the global enterprise GPU market share, and the second-ranked AMD market share will only be 8.5%. From PC game graphics acceleration to data center, how has the GPU developed?
A GPU is an accelerator that contains a large number of parallel computing units and is good at floating-point operations. It can not only meet the needs of 3D graphics rendering, but also meet the computing needs of HPC and AI. In 2007, Stanford computer science professor Ian Buck developed CUDA — a platform for programming GPUs. It may not be simply considered that CUDA has brought GPU beyond games and professional vision, making graphics cards available for all walks of life; but CUDA has become a learning hotspot for developers around the world, and it is also an important part of Nvidia's HPC, AI and even the Omniverse (Omniverse). )important parts of. At the same time, the prevalence of CUDA has locked developers in the NVIDIA ecosystem.
From the perspective of GPU architecture, the game GPU can accelerate various applications in the data center by removing some graphics rendering fixed-function units. However, Nvidia clearly regards games and data centers as two independent businesses at the operational level. Although the chip architectures of the two types of GPUs are similar, their markets, customers, and application directions are completely different. At the same time, as the business status of the data center becomes more and more important, its GPU also has some characteristics of its own, and even this will affect the graphics GPU. For example, the Hopper architecture released by Nvidia last year focuses on the addition of the Transformer engine and focuses on NLP (Natural Language Processing).
From a broad classification point of view, the data center GPU belongs to the accelerator category, which is an accelerator around the CPU. From the perspective of the value ratio of different devices and chips inside the data center server, the value ratio of the accelerator is far less than that of the CPU. The previous data given by Ark Invest is that in 2020, the CPU accounted for 83% of the value of all processor chips, which means that the value that the GPU can obtain is less than 17%.
Figure 2: 2020-2030 Value Proportion of Devices in Servers (Unit: USD) Source: Big Idea 2021, Ark Invest
It can be seen from the difference in revenue scale between Nvidia and Intel. Even though Intel is facing huge market and competitive pressures, the company's DCAI (data center and AI) business revenue in the past year still far exceeds Nvidia's data center business revenue. This means that CPUs are more profitable than GPUs, but that may soon change. Ark Invest believes that within 10 years after 2020, the value of CPU in data center servers will drop sharply to 40% (Figure 2).
If CPU, storage, accelerator and other categories are all taken into account, the accelerator category market value is expected to reach $41 billion by 2030, and the CPU market value will shrink slightly to $27 billion. Ark Invest said: "Accelerators including GPUs, TPUs, FPGAs, etc. will be suitable for applications that require computing performance, such as AI, data analysis, drug discovery, cloud games, etc."
Of course, "accelerator" does not just refer to GPU, it also includes FPGA, AI chips and various ASICs. But "in the next five years, GPU will continue to dominate the accelerator market, which is related to its programmable and software stack characteristics." Valuates Reports' data center accelerator market report also mentioned that GPUs accounted for 85% of the overall data center accelerator market in 2018.
Regarding the growth space and market value data of the data center accelerator market, the magnitude of the data given by different analysis agencies is very different, and the CAGR (compound annual growth rate) data is also varied. However, all forecasts believe that in the next ten years, the CAGR of this market will maintain double-digit growth. For example, Ark Invest previously predicted that data center accelerators will grow nearly seven times during 2020-2030.
A new round of "a hundred flowers blooming"
In most subfields of the electronics industry, there are many market participants only in the early stages of development, and the entire industry will show the characteristics of "a hundred flowers blooming". Only with huge room for growth will the market attract many players to enter.
At this stage, no matter from which point of view, the data center accelerator market is quite attractive. In other words, the GPU has transformed from a PC graphics rendering accelerator to an accelerator for various applications in the data center. This type of chip has expanded from a highly mature field to a new field with huge room for development.
Even if the new players in this market will face giants like Nvidia, there are still companies willing to participate in this market. There are two main reasons for this: On the one hand, it is related to Sino-US trade frictions and the trend of regionalization of the global semiconductor market. China has a more urgent demand for chip autonomy than other regions in the world. For new entrants in China, No matter from the perspective of market or technology, GPU is relatively easier to enter than CPU; on the other hand, this market is indeed attractive enough.
In fact, not only newcomers have shown considerable enthusiasm, but even Intel's giants are also working hard on the data center GPU business. In the past year, Intel has spared no effort in the market promotion of Xe-HPC architecture graphics cards. The value of CPUs in the data center market has been declining. Intel urgently needs to seek opportunities for business expansion in the data center market.
In addition to the data center market, some companies are also trying to enter the highly mature PC graphics card market that has been controlled by Nvidia and AMD. In most cases, the products in these markets are by-products brought about by the data center market. After the GPU architecture becomes modular and elastically scalable, this attempt is technically feasible. Of course, companies such as Innosilicon and Moore Threads in China have laid out PC graphics cards because of the background of demand for semiconductor independence and localization.
Among the newcomers, Imagination is a very representative one. In the 1990s, Imagination was also one of the companies competing in the graphics accelerator card market. Later, the company's main business shifted to the mobile market. Imagination is called a "new market player" because its GPU business has not been involved in the data center market before.
After the private equity fund Canyon Bridge announced the acquisition of Imagination in 2017, the history of Imagination opened a new chapter. Imagination's current GPU design is based on a modular design scheme, which scales through the core of the same architecture, covering various applications from mobile phones to data centers-this is a solution that is very much in line with contemporary GPU design concepts.
"From an IP perspective, we think the key to this transition is ensuring that the hardware can be built in a scalable way to allow more performance headroom for years to come." David Harold, chief marketing officer, Imagination Technologies In an interview with "International Electronic Business" analysts said: "Whether it is now or in the future, PC and data centers are important markets for Imagination. We recognize that mobile 5G applications are driving a significant increase in the number of connections, which is also important for qualified , Reliable cloud solutions put forward higher requirements. Based on the data center configuration of the CXT architecture, we are helping customers integrate ray tracing technology into cloud solutions in a sustainable and efficient manner. The C series has already Provide authorization in this market, and plan to launch corresponding chips within the next 18 months."
In addition, Bai Nong, chairman of Imagination China, mentioned at the "2022 Global CEO Summit" that for Imagination, China's cloud game market is one of the four rapidly developing markets. So the "data center" GPUs the company is eyeing include graphics rendering, which is Imagination's forte. “We believe that GPUs will be a fundamental component of data center systems, not only because of their 3D graphics processing capabilities, but also because of their highly parallel computing capabilities. In particular, the latter is essential in one-to-many scenarios tool."
Last year, Sun Erjun, co-founder and CMO of Muxi, mentioned in an interview with "International Electronic Business": "Using general computing as an entry point takes into account the urgent market needs in scientific computing, machine learning and AI training. The new The enhancement of some rendering scenes also puts forward new requirements for the computing part, and even rendering and computing are being closely integrated.” In terms of application, Mu Xi’s GPU involves artificial intelligence, smart cities, data centers, and cloud computing , autonomous driving, life sciences, digital twins, metaverse, etc.
The general environment of social digital transformation is itself based on the development of mobile phones, IoT, AI, and big data, as well as their demands for IT, communications, and data center accelerators. Just like the application direction listed by Mu Xi, GPU has a wide and huge market potential. Driven by the huge market potential, new entrants have disclosed the dynamics of GPU products. For example, Hanbo Semiconductor announced at the "WAIC 2022 World Artificial Intelligence Conference" that it will launch GPU products; Mu Xi also said that the company will launch GPU products in the future. The GPU capable of rendering can fully cover the high-performance GPU market; Biren Technology released the first general-purpose GPU chip BR100, and its own original architecture...
GPU market segmentation, and AI fever
Last year, Nvidia founder and CEO Jensen Huang divided data centers into six categories: supercomputing centers, enterprise computing data centers, hyperscalers, cloud computing data centers, AI factories, and edge data centers. Seeking Alpha has done a detailed analysis of these six sectors, and most of the large chips for data centers can be classified according to this.
Supercomputing Data Center
In 2021, 70% of the world's TOP 500 supercomputers use Nvidia GPUs, and the adoption rate of Nvidia GPUs in new systems is still increasing. Applications of this supercomputing data center include quantum computing, climate prediction, fossil energy extraction, molecular modeling, physical simulation, aerodynamics, nuclear fusion research, etc.
Enterprise Data Center
An enterprise data center is a data center operated within an enterprise for projects such as IT, finance, medical care, or customer data support. The capacity of this field is very large, and it is also the focus of current market competition.
super cluster
Previously, IDC defined a super cluster data center, which is a data center with an area of more than 10,000 square feet and more than 5,000 servers. Its infrastructure is large in scale and can be flexibly scaled to meet the needs of different customers. Tech giants such as Meta, Google, and Alibaba all have such data centers.
cloud computing data center
Cloud computing data centers are infrastructures that provide services through the cloud, with applications such as cloud gaming, automated customer service, and advanced medical imaging. Canalys data shows that Nvidia's cloud computing data center market share will reach 33% in 2021.
AI factory
AI factory is a new type of data center. Some enterprises begin to pay attention to the utilization of data, and use AI to optimize supply chain, predictive maintenance and process control of manufacturing. The production line and supply chain of BMW, one of Nvidia's typical customers, has fully utilized robots, AI, and digital twins in AI factories.
Edge Data Center
The edge data center is a relatively small data center that is closer to end-side users, achieving low latency and high speed of data transmission. This type of data center has a wider range of applications, such as warehouses, retail, automobiles, robots, and smart transportation.
In these six categories, Nvidia can provide chips and system/solution products horizontally, and can also provide bottom-up full-stack software vertically. However, the author believes that although Nvidia has achieved a first-mover advantage in certain markets with OEMs or partners, new players still have great opportunities in potential markets.
It is worth mentioning that in terms of application segmentation in different dimensions, AI is the highlight of GPU. The upgrade focus of Nvidia Hopper's new architecture is also clearly biased towards AI, which has a lot to do with the current AI fire. According to Omdia data, in the category of AI processors alone, Nvidia GPUs will account for more than 80% of global AI processor revenue (excluding CPUs) in 2021, far exceeding market participants such as AMD, Google, and Intel.
The market demand for AI chips is also rising. Over the years, both start-up companies and established semiconductor industry companies have tried their best to strengthen the layout of AI chips-in a broad sense, GPU is also a branch of AI chips. Ark Invest's forecast shows that from 2021 to 2025, the data center's spending on AI processors (including GPUs, but excluding CPUs) will increase fourfold, from $5 billion to $22 billion.
Markets outside of Nvidia
According to data from statistical agencies, there is still room for multiple growth in data center GPUs, and Nvidia only occupies a part of the potential market. Nvidia's market position in AI training is almost unshakable, but in 2021, Cassell and the Omdia team mentioned in a report that other chip manufacturers will swallow Nvidia's market share in the next few years because of the market acceptance of other types of AI chips. The degree is getting higher and higher.
The report pointed out that it is expected that by 2026, the share of GPUs in the direction of AI applications will drop to 54%, and this value will be as high as 82% in 2021. In terms of AI chips, GPU temporarily occupies the bulk of the market value. With the emergence of many AI chip companies around the world, more FPGAs, TPUs and various AI-specific chips will appear in the future.
Therefore, Nvidia is facing competitive pressure in data center acceleration chips. After all, this market is far from mature and may be disrupted in the future. Even so, in the past interviews with analysts of "International Electronic Commerce", most market participants still said that they feel great pressure on the existence of Nvidia.
In 2021, when Hanbo launched the AI reasoning chip, it said that Nvidia GPU already has considerable ecological advantages on the AI training track, so the entry point for Hanbo to enter the market is the AI reasoning chip in the direction of DSA. In the cover story interview of the October 2021 issue of "International Electronic Business" magazine, Graphcore admitted frankly that as a data center AI chip company, the company is working hard to expand the application field, and is focusing on the Internet, finance, and research based on the existing human and material resources. , medical health and the other five parts cannot cover such a wide ecological range like Nvidia.
At the annual GTC conference, Nvidia announced that it has updated some software libraries in a certain field, and the corresponding performance has been improved by x times, which is increasing the difficulty of the GPU track in the data center.
But why are there still so many companies wanting to enter this market? Sun Erjun said: "GPUs have a large number of application scenarios in key fields such as artificial intelligence, scientific computing, finance, and industry. Compared with dedicated AI chips, GPUs are more versatile, richer in landing scenarios, and wider in application fields. A large amount of capital The influx into the GPU track can reflect that the market is generally optimistic about GPU.” At the same time, he also mentioned the thrust of the Chinese government’s policies, especially the “East Counts West” project, and the prospect of the digital economy.
Last year, when Hanbo announced that it was going to make a GPU, the analyst of "International Electronic Business" asked Qian Jun, the founder and CEO of Hanbo Semiconductor, "Why did you choose to enter such a more stressful track?" Qian Jun replied: "In the past , Hanbo focuses on the computing and processing of video, but now we want to do 'pixel generation', we will first select some industries and fields to do it like we did AI in the early stage."
Imagination also has a market-specific observation assumption. David said: "Heterogeneous computing is on the rise, and more and more companies are making customized chips." This is a market opportunity for data center GPUs outside of Nvidia. This may have something to do with Imagination's role as an IP provider.
The data center market value has shifted from CPU to accelerator, reflecting the helplessness of Moore's Law stagnation. CPU, a processor that relies more on Moore's Law to move forward, is facing the embarrassment of being unable to meet the needs of the rapid development of the digital economy in the direction of HPC, so accelerators with highly parallel data processing capabilities have become hot.
GPU chips are still a newborn in the data center market, and the competition for data center GPUs has just started. Nvidia temporarily has an advantage in this round, but this does not mean that other market participants have no chance of winning.
END
Welcome to join Imagination GPU and artificial intelligence communication group 2
Join the group, please add the editor WeChat: eetrend89
(Please add company name and title)
recommended reading
Dialogue with the Chairman of Imagination China: Using GPU as the fulcrum to strengthen software and hardware collaboration to facilitate digital transformation
[Video] Technical Director of Imagination China fully interprets IMG DXT GPU
Imagination Technologies is a UK-based company dedicated to the research and development of chips and software intellectual property (IP). Products based on Imagination IP are used in the phones, cars, homes and workplaces of billions of people around the world. For more information on cutting-edge technologies such as the Internet of Things, smart wearables, communications, automotive electronics, and graphics and image development, welcome to Imagination Tech!