Heaptalk, Jakarta — Nvidia officially introduced its latest AI chip called H200 at the SC23 Conference in Denver, Colorado, USA (11/13). The H200 Tensor Core GPU is equipped with faster and larger memory, HBM3e.
The company claimed that this AI chip can fuel the acceleration of generative AI and large language models while advancing scientific computing for high-performance computing (HPC) loads. With HBM3e, the Nvidia H200 provides 141GB of memory at 4.8 terabytes per second, almost double the capacity and 2.4x more bandwidth than its predecessor, the Nvidia A100.
According to Vice President of Nvidia’s Hyperscale and HPC Computing Business Ian Buck, large amounts of data must be processed efficiently at high speed using large and fast GPU memory to create intelligence with generative AI and HPC applications.
“With Nvidia H200, the industry’s leading end-to-end AI supercomputing platform just got faster to solve some of the world’s most important challenges,” said Buck in the company’s official statement (11/13).
The AI chip will start shipping in Q2 2024
Nvidia is optimistic that this AI chip will lead to further performance leaps, including nearly doubling inference speed on Llama 2, a 70 billion-parameter LLM, compared to the H100. The company will add more improvements to the chip in future software updates.
H200-powered systems from server manufacturers and cloud service providers are expected to begin shipping in Q2 2024. Amazon Web Services, Google Cloud, Microsoft Azure and Oracle Cloud Infrastructure will be among the first cloud service providers to deploy H200-based instances starting next year, in addition to CoreWeave, Lambda, and Vultr.
Further, H200 can be deployed in every type of data center, including on-premises, cloud, hybrid-cloud, and edge. A number of Nvidia’s global ecosystem of partner server makers can update their existing systems with an H200, spanning ASRock Rack, Asus, Dell Technologies, Eviden, Gigabyte, Hewlett Packard Enterprise, Ingrasys, Lenovo, QCT, Supermicro, Wistron, and Wiwynn, can update their existing systems with an H200.
