NVIDIA H200

H200 is NVIDIA’s HBM upgrade to H100.¹ It competes against MI325X.

Specifications

H200 GPUs are the same as H100 GPUs (SXM5 or GH variant?), just with 144 GB of HBM3e (more and faster HBM). And maybe more power? Not sure.

Performance

I’m pretty sure the H200 performance table is the same as the H100 performance table since both have 132 SMs for the SXM5 variant.

The $R_{\text{peak}}$ per H100 and H200 varies between 55.0 and 59.0 TF/GPU which is somewhere between the VFMA and Matrix FP64 performance above.

Form factors

H200 is sold on an 8-way SXM5 HGX baseboard just like H100 (NVL8), on a GH200 superchip, or integrated into a DGX GH200 pod (NVL32).²

Footnotes

1. DGX GH200 for Large Memory AI Supercomputer | NVIDIA ↩

2. One Giant Superchip for LLMs, Recommenders, and GNNs: Introducing NVIDIA GH200 NVL32 | NVIDIA Technical Blog ↩