NVIDIA announces H200 GPU with better memory for generative AI
NVIDIA has announced a new enterprise-grade GPU with improved AI computing capabilities for generative AI and large language model workloads. The upcoming NVIDIA H200 is the first GPU to boast HMB3e memory, which is faster and has greater capacity than the previous solution used by the NVIDIA H100.
Although the H200 is essentially the same as the H100, aside from the memory upgrade, the improvements enabled by the HMB3e memory are substantial.
The company says that the new memory used by the upcoming H200 GPU offers a bandwidth of 4.8 terabytes per second and a total memory capacity of 141B. In contrast, the NVIDIA H100 GPU achieved a memory bandwidth of 3.35 terabytes per second and 80GB capacity.
Story continues after the video
“With NVIDIA H200, the industry’s leading end-to-end AI supercomputing platform just got faster to solve some of the world’s most important challenges,” said NVIDIA vice president of hyperscale and HPC, Ian Buck.
The NVIDIA H200 will be available in HGX H200 server boards with four and eight-way configurations compatible with the hardware and software of the current HGX H100 systems.
Will H200 availability be a problem?
The US-based GPU manufacturer says the H200 GPU will be shipping in the second quarter of 2024.
With the previous H100 GPU, NVIDIA had issues meeting the high demand (via The Verge). Whether or not the H200 GPU will face the same supply issues is to be determined. However, NVIDIA says it’s working with the “world’s leading server manufacturers and cloud service providers” for a Q2 2024 release.
In other AI news, Samsung has just announced its generative AI and large language model solution, Gauss. It will be used as the basis for new on-device AI for the Galaxy S24 series. One of the first AI-based features available for the Galaxy S24 will enable users to carry real-time phone conversations with people who speak a different language.
