Why NVIDIA GB200 NVL72 Needs Liquid Cooling: A Technical Explainer

The NVIDIA GB200 NVL72 rack packs 72 Blackwell GPUs into a single rack-scale system — a leap in AI compute density that makes previous generations look modest. But with that many powerful chips comes an unavoidable consequence: heat. A single GB200 rack draws well over 100 kilowatts of power, virtually all of which turns into heat that must be removed continuously.

If you are new to high-density AI infrastructure, you may be asking: Why can't we just use bigger fans? This article walks through the physics, the components, and the design choices that make liquid cooling not a luxury for GB200 — but a requirement.

First: What Is the GB200 NVL72 Rack?

Before diving into cooling, it helps to understand what we are cooling. The "NVL72" name refers to NVIDIA's rack-scale design where 72 GB200 GPUs are interconnected using NVLink — NVIDIA's high-speed fabric that lets all GPUs behave like one giant compute node. The rack splits into two types of trays:

• Compute trays (18U): Each tray holds a subset of the 72 GPUs. These are where the actual AI training and inference happens — and where the vast majority of the heat is generated.
• NVL72 switch trays: These hold the NVLink switching hardware that interconnects all 72 GPUs. They generate heat too, but at a lower and more uniform density than the GPU compute trays.

Why does this matter for cooling? Because the two tray types have very different thermal loads — and any practical cooling solution has to address both.

Why Air Cooling Reaches Its Limit

Traditional enterprise servers run at roughly 5–15 kW per rack. A well-designed air-cooled room — with hot-aisle / cold-aisle containment, efficient CRAC units, and carefully sized server fans — handles this comfortably. The GB200 NVL72 rack changes the math entirely.

Air cooling hits a wall for three reasons:

Heat capacity per unit volume

Air carries very little heat compared to a liquid like water or a glycol-water mixture. To move the same amount of heat with air, you need massive airflow — which means larger fans, more fan power, and louder, less efficient systems. At GB200 rack densities, the fan energy itself becomes a significant fraction of the IT load, erasing any efficiency gains.

Hotspot concentration

Modern GPUs do not heat up uniformly across their surface. The highest-power compute blocks concentrate heat into small areas — "hotspots" that can run significantly hotter than the average package temperature. Blowing cold air over a whole chip cannot target these hotspots precisely; by the time air reaches them, it has already picked up heat from surrounding components.

Space envelope

Dense GPU trays leave little room for large heat sinks, ducting, and airflow paths. Engineers cannot simply bolt a bigger fan on top — there is no space. Liquid cooling sidesteps this by moving heat transfer directly onto the chip surface and carrying it away through pipes.

How Direct Liquid Cooling Works for GB200

The cooling architecture used for GB200 is called direct-to-chip liquid cooling. The basic loop is straightforward:

1. A cooling distribution unit (CDU) — either rack-mounted or room-level — supplies conditioned coolant at a controlled temperature and pressure.
2. The coolant flows through a manifold mounted on the tray, which distributes it evenly to each cold plate.
3. Each cold plate sits directly on top of a GPU (or other hot component). Coolant flows through microchannels inside the plate, absorbing heat from the chip.
4. The now-warmer coolant returns through a separate manifold to the CDU, where its heat is rejected to the facility's water loop or to ambient air.

The key idea: instead of moving air across the chip and hoping enough heat reaches the airflow, liquid is brought right to the heat source. This is far more efficient and allows much higher heat flux densities.

The Components, Explained

If you are evaluating a GB200 cooling solution, you will run into four core components repeatedly. Here is what each one actually does — and why each matters.

Cold Plate

A cold plate is a metal (typically aluminum or copper) block mounted directly on the GPU package. Inside are microchannel structures that create turbulent, high-contact coolant flow. Its job is to transfer heat from the GPU die into the coolant. The performance of a cold plate depends on its internal channel geometry, the thermal interface material (TIM) between plate and chip, and how flat the mounting surface is. A badly designed cold plate — even if it looks identical from the outside — can leave the GPU running hot while the coolant itself stays cool.

Manifold

A manifold is the distribution block that feeds coolant to multiple cold plates on a tray — and collects the warmed coolant back. The engineering challenge here is flow balance: you want every GPU to receive roughly the same coolant flow rate, so no GPU starves for cooling. Poor manifold design leads to uneven temperatures across the tray, which forces the whole system to run conservatively to protect the hottest chip.

Quick Disconnect (QD) Fittings

In a 72-GPU rack, trays need to come out for service, upgrades, or replacement. Quick-disconnect fittings let you disconnect a tray from the liquid loop without draining the entire system and without spilling coolant on the electronics. Both sides of the fitting self-seal when separated. QD quality directly affects reliability — a leaking or sticky QD in a production rack is a real operational headache.

Inlet / Outlet Ports

These are the interface points where the tray's cooling loop connects to the rack's manifold and, ultimately, to the CDU. They define the flow rate, pressure drop, and connection standard for the whole system. Mismatched ports between trays, manifolds, and CDU are a common integration mistake.

Compute Tray vs. Switch Tray: Why They Are Cooled Differently

Not every tray in a GB200 rack generates the same amount of heat — and a well-designed cooling solution reflects that.

• Compute trays: These hold the GPUs. Heat density is extreme and concentrated on each chip. Direct liquid cooling with cold plates on every GPU is the practical option.
• Switch trays: These hold the NVLink switching ASICs. They generate heat, but at lower and more distributed density. Depending on the design, they can sometimes be cooled with air, a lighter liquid solution, or a hybrid approach — which is why cooling kits often offer an optional air-cooling package for the switch tray specifically.

Understanding this distinction matters because a one-size-fits-all cooling kit either over-engineers the switch tray (wasting cost and pressure drop) or under-engineers the compute tray (leaving GPUs thermally constrained).

What This Means If You Are Planning a GB200 Deployment

If your team is procuring or building GB200-based systems, the cooling loop is not an afterthought — it is part of the system design. A few practical takeaways:

• Plan cooling infrastructure in parallel with server procurement. The CDU, facility piping, coolant type, and water temperature setpoints need to be sorted out before trays arrive.
• Tray-level cooling design is its own workstream. Cold plates, manifolds, and QDs need to be selected early — not after the rack shows up.
• You do not necessarily have to design everything from scratch. Pre-engineered cooling kits matched to the GB200 tray form factor exist, which can cut months of custom development.
• Think about serviceability. QD fittings and manifold layouts affect how easily trays can be removed and replaced — an operational concern that shows up only after deployment.

About ALVC

ALVC engineers thermal management solutions for AI infrastructure, power electronics, and energy applications. Our standardized cooling kit for the NVIDIA GB200 platform covers both the compute tray (cold plate, manifold, inlet/outlet, QD) and the switch tray (with an optional air-cooling package) — available starting from 1 unit for teams that want to skip the custom thermal design cycle.

Learn more about ALVC's thermal engineering services →