The first is the first H100 NVL graphics card. Compared with the H100 PCIe graphics card, the memory is increased to 96GB HBM3e, which is composed of two H100 PCIe and interconnected through three NVLink 4. That is, the H100 NVL has 188GB of video memory, and the memory interface width is 6144 bits. 7.8TB/s, connected by two PCIe 5.0 x16 interfaces, that is, four slots, power consumption 700W
The performance of the H100 NVL is higher than that of two H100 SXMs, and it is a full-blooded GH100 core, but the problem is that the H100 NVL is composed of two cards, each with 96GB of video memory, but the two cards together only 188GB of VRAM instead of 192GB
Although there are two cards, the power consumption remains the same, which is 700W like a single H100 PCIe. In this case, the frequency of the H100 NVL is higher than that of the H100 PCIe, so the performance is still improved when the power consumption is low.
Then the server configuration of 4 specifications is updated.
The first is the L4-120X configuration, which is composed of 8 L4 graphics cards. The performance is roughly equivalent to RX 6800, dual-slot CPU, a single L4 is 24GB video memory, 72W power consumption, and bandwidth 300GB/s. Equipped with a dual-slot CPU, other parameters are unknown for the time being.
Then it is the L40 configuration, with 8 L40 graphics cards, the performance of a single block is about 10% higher than that of the RTX 4090. It is also equipped with a Connect-X7 network card, and other configurations are unknown. Then it is the H100 NVL
configuration, with 4 One H100 NVL (8 graphics cards), the performance of a single block is about 11% higher than that of RTX 4090, and it is equipped with a Connect-X7 network card. Other configurations are unknown. This should be the best configuration at present
. Two BlueField-3 DPUs and two Grace Hoppers (H100+Garce CPU), other configurations are unknown.
Then there is the new NVIDIA OVX configuration, equipped with four L40 graphics cards and one BlueField-3 DPU, equipped with dual-slot CPUs, other unknowns
Other content is the re-introduction of DGX H100, Grace SuperChip (two Grace CPUs), Grace Hopper (Grace CPU+H100), and BlueField-3 DPU at GTC 2023
Then there is the H800 graphics card, which is a product with half the memory bandwidth of the H100, and the interconnection rate is limited to 300 GBps. The H800 graphics card is exclusively for the domestic version.
Regarding the software or other parts, this article does not mention it. If you want to know, you can read this: Nvidia GTC 2023 Keynote Live Blog