AMD MI325X

MI325X is an AMD GPU built on CDNA 3 that was launched on October 10, 2024 and will be in production in 4Q2024.¹

It is positioned to compete against H200 and, just as H200 was a memory bump to H100, MI325X is a memory bump to MI300X. However, MI325X also has a 1000 W maximum power, whereas MI300X was 750 W maximum.

Specifications

Each MI325X GPU has:²

• 8 XCDs
  • 304 CUs (? per XCD)
    • 19,456 Stream Processors (64 per CU)
    • 1,216 Matrix Cores (4 per CU)
  • 2.1 GHz (peak)
  • 2:4 structured sparsity
• 256 GB HBM3E (8 stacks?)¹
  • 6 TB/s (max)
• 7 AMD Infinity Fabric (D2D)³
• 1 PCIe Gen5 (H2D)³
• 1000 W maximum (TBP)

Its compute subsystem (XCDs and below) appear identical to MI300X, but its memory subsystem is upgraded to HBM3E. It is also specified at 1000 W TBP instead of 750 W.

Performance

The launch advertised 1.3 PF of FP16 and 2.6 PF of FP8.¹ This appears to be the same as MI300X.

The following are theoretical maximum performance in TFLOPS:²

Data Type	VFMA	Matrix	Sparse
FP64	81.7	163.4
FP32	164.4	163.4
TF32		653.7	1307.4
FP16		1307.4	2614.9
BF16		1307.4	2614.9
FP8		2614.9	5229.8
INT32
INT8		2614.9	5229.8

Footnotes

1. https://x.com/patrickmoorhead/status/1844415678918603090?s=46&t=UheQB0-0TrbsB2KvmT7aWQ ↩ ↩² ↩³

2. AMD Instinct™ MI325X Accelerators ↩ ↩²

3. The AMD spec page says there are 8 Infinity Fabric links and the GPU supports PCIe Gen5, but it doesn’t specify how those are related. However, the MI300X spec says there are 7x16 Infinity Fabric links and 1x16 PCIe Gen5, so I assume MI325X is the same. ↩ ↩²