NVIDIA GH200

GH200 is NVIDIA’s “Grace Hopper Superchip” which combines a single Grace CPU with a H100 or H200 GPU.¹

The GH200 platform also supports 18x NVLink 4 lanes out of each Hopper GPU, allowing for two GH200 superchips to be coherently connected (forming “GH200 NVL”) or up to 256 GH200 superchips using NVLink Switch.¹

Performance

I haven’t read it, but Torsten Hoefler’s group published a paper on Alps that contains benchmark results for GH200: “Understanding Data Movement in Tightly Coupled Heterogeneous Systems: A Case Study with the Grace Hopper Superchip”

Footnotes

1. https://resources.nvidia.com/en-us-grace-cpu/grace-hopper-superchip?ncid=no-ncid ↩ ↩²