Search AI Chips News Today

BUYER’S GUIDE

Power and cooling considerations for AI deployments

Direct-to-chip liquid-cooled server rack manifolds and coolant piping

Modern AI accelerators draw substantial sustained electrical power and reject concentrated heat flux into your data centre. As socket thermal design power (TDP) scales from 700 W on Hopper to 1,200 W–1,400 W on Blackwell Ultra and AMD Instinct MI355X — and past 2,000 W on next-generation Vera Rubin modules — facility power distribution and thermal rejection have become the primary deployment bottleneck.

1. Matching rack power density to the right cooling technology

Before signing a server purchase order or colocation lease, map your planned per-rack kilowatt envelope against three thermal architectures:

  • Up to 35 kW–45 kW per rack (High-Velocity Air + Rear-Door Heat Exchangers): Suitable for 2 to 4 air-cooled 8-GPU servers per cabinet using hot-aisle containment or chilled-water rear-door coils. Fan power overhead can consume 10% to 15% of total IT load.
  • 60 kW to 145 kW per rack (Direct-to-Chip Single-Phase Cold Plates + Air Assist): The current industry standard for dense 8-GPU OAM baseboards and NVIDIA GB200/GB300 NVL72 racks. Copper micro-channel cold plates capture 75% to 85% of rack heat directly from GPUs, CPUs and NVLink switches, while low-RPM chassis fans cool PCIe NICs, DIMMs and power shelves.
  • 150 kW to 250+ kW per rack (100% Liquid-Cooled Rack Architectures): Engineered for 227 kW Vera Rubin NVL72 cabinets and ultra-dense custom pods. Cold plates cover compute, memory and 1.6T optical switch cages, supported by high-flow Coolant Distribution Units (CDUs) and warm-water ASHRAE W32–W45 loops that enable year-round chiller-free dry cooling in most climates.
Rack Power Tier Cooling Method Typical Coolant Supply Temp Facility Infrastructure Requirements
20 kW – 40 kW / Rack Containment Air / Rear-Door HX 18°C – 27°C Air / 15°C Water Three-phase 415V PDUs, raised floor or overhead busway
80 kW – 142 kW / Rack Direct-to-Chip Cold Plate (PG25) + Air 25°C – 35°C (ASHRAE W32) In-row or in-rack CDU, dripless blind-mate QDs, 54V DC rack busbars
150 kW – 230+ kW / Rack Full Rack Liquid Cooling + BBU Shelves 32°C – 45°C (ASHRAE W40/W45) High-capacity facility headers, ~300+ L/min flow per pod, supercapacitor/BBU smoothing

2. Mitigating synchronous GPU power transients

When thousands of GPUs start or finish an AllReduce collective step simultaneously, cluster power draw can swing from 20% idle to 100% peak load in milliseconds. Verify whether your rack specification includes in-rack battery backup units (BBUs) or capacitor energy-storage shelves to smooth di/dt power steps and protect upstream utility transformers.

Explore detailed facility engineering coverage on our Data Centres hub, continue to Choosing the right interconnect for AI workloads, or return to the AI Chip Buyer’s Guide.