Memory bandwidth

Memory bandwidth is expressed in GT/s (gigatransfers per second). The bytes per transfer is a function of the width of each channel and how many channels are used in parallel.

For example, Fugaku has four stacks of HBM2 and a total of 1,024 GB/s of memory bnadwidth per CPU.¹ This breaks down to 256 GB/s/stack which can be calculated by first knowing that HBM has 128-bit channels and 8 channels per stack:

$128\text{ bits}\times 8\text{ channels}=128\text{ bytes per transfer}$

The HBM is clocked at 1.0 GHz, and because HBM is DDR, there are two transfers per clock. Thus, HBM has $2\times 10^9\text{ transfers/sec}$. Given 128 bytes per transfer,

$2\times 10^9\text{ transfers/sec}\times 128\text{ bytes/transfer}=256\times 10^9\text{ bytes/s}=256\text{ GB/s}$

Memory channels are the interfaces with which memory controllers can talk to memory, and a single memory chip (die) may implement one or more channels.²

DDR

DDR5 has 64-bit channels, but unlike previous generations, implements these as two independent 32-bit sub-channels. RDIMMs have an additional 8 bits for ECC.³

GDDR

Each channel of GDDR6 uses two 16-bit subchannels.⁴

Memory	Data width per channel	Typical number of channels
DDR5	32 bits	2 channels
GDDR5	32 bits	16 channels
HBM2e	128 bits	8 channels
HBM3	64 bits	16 channels
LPDDR5	16 bits

HBM

HBM2 uses 128-bit channels, and each stack has eight channels.⁵

HBM3 gets complicated:

• It uses 64-bit channels and sixteen channels per stack
• Each 64-bit channel is implemented using two 32-bit subchannels (“pseudo-channels”).⁶

LPDDR

LPDDR5 uses 16-bit channels.²

Footnotes

1. スパコンを使った 最先端の天気予報研究 (tsukuba.ac.jp) ↩

2. LPDDR5 Tutorial - Deep dive into its physical structure - systemverilog.io ↩ ↩²

3. DDR5 Memory Standard: An introduction to the next generation of DRAM module technology - Kingston Technology ↩

4. Micron Reveals GDDR6X Details: The Future of Memory, or a Proprietary DRAM? | Tom’s Hardware (tomshardware.com) ↩

5. 7.1. High Bandwidth Memory (HBM2) DRAM Bandwidth (intel.com) ↩

6. What is High Bandwidth Memory 3 (HBM3)? | Synopsys ↩