Liquid cooling: rear-door heat exchangers and in-row units

Between pure air cooling from the room's perimeter and liquid flowing through cold plates on the chips lies a family of systems that bring water close to the racks while the servers themselves stay air-cooled. They are often the first liquid cooling a technician meets, because they can be added to an existing air-cooled hall. Hall A has two of them: a row of rear-door heat exchangers on the cabinets of a customer whose racks grew from 6 kW to 25 kW, and a pod of in-row cooling units around a small high-performance cluster. Carmen Ortiz, the Corvid AI site engineer from Hall D, calls them "liquid cooling with training wheels." This lesson explains how they work, how to inspect them and what can go wrong.

Why bring the water closer

Perimeter cooling units must move air across the whole hall, through the floor or the room, to reach the racks. As rack loads grow, the volume of air becomes impractical, as Lesson 6 showed. Bringing the cooling coil to the rack shortens the air path to almost nothing. The servers still blow their exhaust out of the back, but a coil a few centimetres away absorbs the heat before the air ever reaches the room. Close-coupled cooling, as it is called, can handle racks of 20 to 50 kW, and some designs more, while the servers remain standard air-cooled models.

Rear-door heat exchangers

A rear-door heat exchanger, RDHx, replaces the back door of a cabinet with a door that contains a finned coil. Chilled water flows through the coil from hoses connected at the bottom or top of the door. The servers' exhaust passes through the coil and leaves at close to room temperature, so the cabinet adds little or no heat to the room.

• Passive rear doors have no fans. The server fans alone push the air through the coil. They suit moderate loads and add little pressure that the server fans must overcome.
• Active rear doors have their own fans, which help push the air through a deeper coil and handle higher loads. The door's fans are controlled to match the servers' airflow.

A rear door's capacity depends on the water temperature, the water flow and the airflow from the servers. Warmer water means less capacity. Rear doors are often supplied with water above the room's dew point, so the coil never sweats, which means a condensate drain is not needed. Water from a chilled-water plant at 18 °C (64 °F) is well above the dew point of a hall kept at moderate humidity.

Rear doors are hinged like ordinary cabinet doors, so the servers can be reached from behind. The hoses must be long enough and routed so that the door can swing open without stressing them, and the door must be fully closed and latched for the coil to work: with the door ajar, hot air escapes around the coil into the room.

# Rear-door heat exchanger: share of a cabinet's heat removed versus supply water temperature
rack_kw = 25.0
exhaust = 42.0               # server exhaust air entering the coil
effect = 0.80                # coil effectiveness at this airflow (illustrative)
for water in (16, 18, 20, 22):
    leaving_air = exhaust - effect * (exhaust - water)
    removed = rack_kw * (exhaust - leaving_air) / (exhaust - 24.0)
    removed = min(removed, rack_kw)
    print(f"water {water} C: air leaves the door at {leaving_air:4.1f} C, door removes {removed:4.1f} of {rack_kw:.0f} kW")

water 16 C: air leaves the door at 21.2 C, door removes 25.0 of 25 kW
water 18 C: air leaves the door at 22.8 C, door removes 25.0 of 25 kW
water 20 C: air leaves the door at 24.4 C, door removes 24.4 of 25 kW
water 22 C: air leaves the door at 26.0 C, door removes 22.2 of 25 kW

The output shows the trade-off the facilities team made for the customer's 25 kW cabinets. With water at 18 °C (64 °F) the doors remove almost all the heat. On the few days when the plant runs warmer water, a little heat reaches the room, and the perimeter CRAHs pick it up.

In-row cooling units

An in-row cooling unit is a narrow cooling unit the height and depth of a rack, placed in the row between racks. It draws hot air from the hot aisle through its back, passes it over a chilled-water coil or a refrigerant coil, and blows cool air out of its front into the cold aisle, a short distance from the server inlets. Several in-row units are spread along a row, typically one for every two to four racks, depending on the load.

In-row units work best with containment. In Hall A's high-performance pod, eight racks of about 30 kW and five in-row units form a closed hot aisle: the racks and units stand back to back around it, with doors at each end and a roof. Every unit draws from the same enclosed hot aisle, and if one unit fails, the others draw more air to compensate.

In-row units are controlled on the temperature of the air they supply or of the rack inlets nearby, and their fans on the pressure difference across the containment or on the hot-aisle temperature. Like perimeter units, they share load and run with at least one spare.

import math
# In-row units needed for a pod, with one spare (N+1)
racks = 8
rack_kw = 30
unit_kw = 70                  # rated capacity of one in-row unit at the pod's air and water temperatures
load = racks * rack_kw
n = math.ceil(load / unit_kw)
print(f"Pod load {load} kW; units needed N = {n}; installed N+1 = {n + 1}")
for failed in (0, 1):
    running = n + 1 - failed
    print(f"  {failed} failed: {running} units share the load at {load / running:.0f} kW each ({load / running / unit_kw:.0%} of rating)")

Pod load 240 kW; units needed N = 4; installed N+1 = 5
  0 failed: 5 units share the load at 48 kW each (69% of rating)
  1 failed: 4 units share the load at 60 kW each (86% of rating)

Four units could carry the pod's 240 kW, so a fifth is installed as the spare. With all five running, each works at about two-thirds of its rating, which keeps fan speeds and noise down. If one fails, the other four run at 86 percent and the pod stays within its limits until the failed unit is repaired.

Water at the rack: hoses, couplings and manifolds

Both rear doors and in-row units bring water into the white space, which means every joint is a possible leak a few feet from live servers. The connection hardware matters:

• Manifolds: headers of pipe running under the floor or overhead along the row, with a supply and a return connection for each door or unit.
• Isolation valves: a valve on the supply and the return of each connection, so a single door or unit can be isolated without draining the row.
• Flexible hoses: braided or rubber hoses that let doors swing and units be moved slightly. Hoses have bend-radius limits and age; they are inspected and replaced on a schedule.
• Quick-disconnect couplings: self-sealing couplings that let a hose be connected or disconnected without spilling. They must be fully engaged; a half-engaged coupling can leak or restrict flow.
• Strainers: fine screens that catch particles before they reach a coil's small passages. A clogged strainer reduces flow and capacity.
• Air vents: small valves at high points to bleed out trapped air, which otherwise blocks flow and causes noise.

Technicians at Stonebridge may inspect all of this, but only those trained and authorized on the site's liquid cooling procedures may operate valves or couplings, as DCT-120 explains. Jamal's job on his rounds is to look, listen, feel for temperature and report.

Leak detection and drip protection

Wherever water runs near IT equipment, the site installs leak detection:

• Sensing cable: a cable laid on the floor or in drip trays under manifolds and units. Water anywhere along its length changes its electrical properties and raises an alarm, and many systems report the distance along the cable to the leak.
• Spot sensors: small probes in drip trays, in the base of in-row units and under manifolds.
• Drip trays and pans: sheet-metal trays under joints and manifolds that catch drips and hold the sensing cable.

Leak alarms go to the monitoring system and to the shift lead's radio. The response procedure is fast and specific: go to the location, confirm what is wet and how much, protect equipment if it can be done safely, isolate the leaking connection if authorized, and call the facilities team. The cause is then found and fixed before the connection is returned to service.

A false alarm, such as a mop left on a sensing cable or condensation in a tray, still requires a visit and a record. Repeated false alarms on the same cable should be investigated, because staff who learn to ignore an alarm will ignore the real one.

Inspection rounds for close-coupled cooling

Roz's checklist for the rear doors and in-row units in Hall A:

• Doors fully closed and latched; no gap around the frame.
• Door fans, on active doors, running and quiet.
• Supply and return hoses: no kinks, abrasion, bulges or stains; supported and clear of door hinges.
• Couplings fully engaged and dry.
• Drip trays dry and clean; leak cable in place and undamaged.
• Supply and return temperatures on the unit or door display within the expected range, and the return warmer than the supply.
• In-row units: front and rear grilles clear; filters clean; no alarms; condensate drain, if fitted, clear.
• Rack inlet temperatures on the monitoring screen normal for the row.

A rear door whose return water is the same temperature as its supply is moving no heat: either the water is not flowing or the door is open. A hand on the door's outlet air tells a lot: it should feel close to room temperature, not hot.

When close-coupled cooling is the right choice

Rear doors and in-row units suit rows or pods of moderately dense air-cooled equipment in halls that already have chilled water. They avoid rebuilding the whole hall. They are less suitable for the newest AI systems, whose heat is concentrated in a few chips at a density no air path can manage; those need direct-to-chip cooling, which Lesson 9 covers. Many sites, including Hall D, combine both: cold plates on the chips, and rear doors or room air to carry the remaining heat from memory, power supplies and network gear.