Cray EX4000 is one of the high-density, liquid-cooled cabinets created by Cray for its Shasta program.
Each EX cabinet has:1
- 8 compute chassis
- 64 compute blade slots total (8 per chassis)
- 4 power shelves with 1 PDU each
- 4 power input whips
- 64 switch slots
Compute blades slot in the front vertically, and switch blades slot in the rear horizontally:
- eight blades can connect to up to eight switches within a single chassis
- hot and cold water connections are in the front
- all Slingshot networking is in the rear
This is the front of an EX4000 cabinet I took at SC19:
You can see the hot and cold water lines (red and blue hoses) and the eight chassis: four stacked vertically on the left and right halves of the cabinet. Each chassis has vertical compute blades into which the cooling hoses are connected.
Here is the back of that same cabinet:
You can see extensive DAC copper cabling that carries the 200G Slingshot between switches within the rack. The teal optical leave the cabinet and connect to other Slingshot groups.
For a list of all the blades compatible with Cray EX, see HPE Cray Supercomputing EX.
Datacenter integration
The EX4000 cabinet has the following physical specifications:2
- 22.1 sqft / 2.05 m per cabinet
- 98” x 46.5” x 68.5” (height x width x depth)
- 2489mm x 1181mm x 1741mm
- Up to 8,000 lb / 3629 kg
- Floor loading up to 362 lbs/sqft
- 400 KVA per cabinet at 480 V or 350 KVA with 400 V
- 4x power shelves per cabinet (one shelf per two compute chassis)
- 8x 15 kW rectifiers configured as 7+1 per power shelf
- 4x PDUs (1 per power shelf)
- 4x whips (3-phase)
Additional specifications for the Cray EX4000 cabinet can be found in the Cray EX Supercomputer QuickSpecs.2