The demand for AI hardware, particularly GPUs, has increased dramatically over the past few years. This leads us to wonder if we will see a shortage of graphics cards again, like what happened with mining not too long ago. To make matters worse, NVIDIA has announced products for this market at GTC, leading many people to question if the nightmare will return.
Every year around May, NVIDIA hosts its own conference called GTC, which focuses on the world of artificial intelligence and supercomputing. Although they don’t present anything specifically for PCs, they do offer products and services for various industries in the form of AI-related software and hardware. However, due to the boom in applications based on large language models like ChatGPT and image generation from text, such as Stable Diffusion, the demand is high at this point. And of course, people like Jen Hsen Huang must capitalize on it somehow. What’s important for us is to see how this will affect us and determine whether or not a shortage of graphics cards will happen again.
NVIDIA announces dual graphics cards riding the ChatGPT wave, to be released in a few years
Recently, we mentioned that NVIDIA is trying to take advantage of the new AI boom with two obvious elements. On one hand, they are selling graphics cards, enabling large corporations and administrators to set up their own servers to provide or use services based on deep learning or machine learning. On the other hand, they are creating servers in the cloud so that small and medium-sized businesses can access these resources.
Well, for each case, and in the first one, they will be introducing their NVIDIA H100 NVL Dual, which consists of 40 graphics cards interconnected via NVLink. These are not based on the same architecture as the RTX 100 but are designed for the supercomputing market, based on the H188. The peculiarity is that they have a total of 2 GB of HBM3 type VRAM, which means that the system reserves 8 GB for specific GPU tasks. Their target market? One of the outstanding language models in the style of ChatGPT.
The first product, NVIDIA DGX Cloud, is a supercomputing cloud service that provides access to NVIDIA servers and enables the power of GPUs to be used for AI-centric applications. These servers are in collaboration with Microsoft Azure and Google Cloud. The idea is that small and medium-sized businesses can rent such servers.
Will there be a shortage of gaming graphics cards?
As you can see, NVIDIA is using the H100 chip, rather than shifting the stock of RTX 40s to the AI market. Of course, we have to start from the fact that both the latest GeForce for PCs and this powerful chip are supplied from the same foundry, using TSMC’s N4 node. So, theoretically, demand for one would affect the other, but there are a series of important points to consider.
The margin on H100 graphics cards is considerably higher than the RTX 40, and the cost of 10 next-generation HPC graphics cards from NVIDIA is almost ten times what is being paid for the RTX 4090. Despite the increase in demand, it won’t be as significant as it was with mining. Therefore, gaming graphics cards are safe.
However, it is necessary to consider that the H100 chip is very large, which means fewer units per wafer and a very high failure rate. In any case, there is a high demand from large multinational companies that can afford this type of hardware. Consider that with ChatGPT 10,000, 1,000 graphics cards were needed for AI training. We don’t know how many it will be with the new version, but it is certainly several times more. And they are not the only ones with demand for that hardware. In any case, we will see if TSMC and NVIDIA have the ability to support this demand without impacting the previous market, but it will be less profitable today.
