Liquid cooling safety: leaks, fluids and pressure

For most of the history of data centers, technicians were taught one rule about water: keep it out of the white space. Hall D breaks that rule on purpose. Each of its 60 compute racks has coolant flowing through cold plates on the processors, supplied by coolant distribution units, CDUs, in the technical galleries, with quick-disconnect hoses at the back of every rack. Liquid cooling is now the normal way to cool high-density AI equipment, and technicians must learn to work safely around pressurized fluid loops beside electrical equipment. This lesson explains the parts of a liquid cooling system, its hazards, the fluids it uses and what technicians do and do not do.

How direct-to-chip cooling works

In Hall D, the cooling system has two loops separated by a heat exchanger in each CDU. The facility water system, FWS, carries water from the building's cooling plant to the CDUs. The technology cooling system, TCS, is the secondary loop that runs from the CDUs through manifolds in each rack to the cold plates on the chips and back. The TCS loop is kept separate so that its fluid can be kept very clean, at a controlled temperature and pressure, and so that a leak in a rack involves only a small volume. Each CDU contains pumps, the heat exchanger, filters, an expansion tank, sensors for temperature, pressure and flow, and controls that react to leaks and alarms.

The TCS fluid in Hall D is a mixture of water and propylene glycol with corrosion inhibitors and biocide, of a kind often called PG25 because it is about 25 % glycol. Other sites use treated water, and some immersion systems use dielectric fluids that do not conduct electricity. Each fluid has a safety data sheet, which technicians can find on the hall's tablet and in the CDU gallery.

The hazards of liquid cooling

• Leaks onto electrical equipment: a dripping fitting or a failed hose can spray coolant onto servers, power shelves or busway, causing short circuits, arcing and equipment damage.
• Pressure: TCS loops typically run at a few bar of pressure; a failed hose or a disconnected fitting under pressure can spray fluid with force into the face and eyes.
• Temperature: return coolant can be warm to hot, 113 °F or more in some designs, and facility water pipes can be cold enough to sweat.
• Slip hazards: glycol mixtures are slippery on smooth floors.
• Chemical exposure: glycol and treatment chemicals can irritate skin and eyes; biocides need careful handling; some dielectric fluids have specific hazards.
• Weight: filled manifolds, CDUs and hoses are heavy; CDUs can weigh hundreds of kilograms.
• Electrical: CDUs contain pumps, drives and controls with their own electrical supplies, which are part of the lockout procedure.

Leak detection and response

Liquid-cooled halls are designed to find leaks fast. Hall D has rope-type leak detection cable under each rack and along every manifold and pipe run, spot sensors in drip trays, and pressure and flow monitoring in every CDU, which can detect a loss of fluid long before any visible puddle forms. The response to a leak alarm depends on its size and place, and is written into the hall's emergency operating procedure. The technician's part is:

• Respond to the location the alarm shows, with the leak response kit from the hall entrance: absorbent pads and socks, a torch, nitrile gloves and safety glasses.
• Look before you touch: from a distance, find where the fluid is and what it is near. If fluid is on or near energized equipment, do not touch it or the equipment; keep back and report it at once.
• If the leak is small and away from electrical equipment, contain it with absorbent socks and pads to stop it spreading toward racks or power.
• Report to the shift lead with the exact location, the size and whether the source is visible. The shift lead decides whether the CDU or the rack's isolation valves are closed, which is done by trained facilities staff or the tenant's engineers under the procedure.
• Mark the spill area as a slip hazard and keep people out.

# Leak alarm triage from the monitoring feed (illustrative rules)
alarms = [  # (sensor, location, wet length ft, near electrical?)
    ("LD-D-14-R", "rack D-14 rear drip tray", 1.0, True),
    ("LD-D-GAL2", "gallery 2 floor by CDU-3", 5.0, False),
    ("CDU-5 FLOW", "loop 2 pressure falling 0.2 bar per 10 min", 0, False),
]
for sensor, where, wet, near_power in alarms:
    if near_power:
        action = "keep back, do not touch, call shift lead now: possible shutdown of rack and loop"
    elif wet > 0:
        action = "contain with absorbent socks, mark the slip hazard, report location and size"
    else:
        action = "no visible leak yet: walk the loop with a torch and report"
    print(f"{sensor:11s} {where:42s} {action}")

LD-D-14-R   rack D-14 rear drip tray                   keep back, do not touch, call shift lead now: possible shutdown of rack and loop
LD-D-GAL2   gallery 2 floor by CDU-3                   contain with absorbent socks, mark the slip hazard, report location and size
CDU-5 FLOW  loop 2 pressure falling 0.2 bar per 10 min no visible leak yet: walk the loop with a torch and report

Many liquid-cooled racks also have automatic responses: a leak sensor in the rack's drip tray can close the rack's supply and return valves and shut down its servers in a controlled way before the leak reaches electrical parts. Technicians should know whether a rack has this feature and must not disable it.

Quick disconnects and hoses

Rack manifolds connect to the servers' cold plates through quick-disconnect couplings, QDs, designed to seal both sides when they are separated so that only a few drops escape. They are the parts most often handled during server swaps and most often blamed for leaks. Rules for handling them:

• Only people trained on the specific rack and QD type connect and disconnect them; at Stonebridge that means Corvid AI's engineers or Stonebridge technicians with Hall D liquid cooling training.
• Follow the server vendor's sequence: usually the server is shut down first, then the QDs are disconnected, return before supply or as the vendor specifies, with an absorbent pad underneath.
• Never force a coupling, use tools on it, or connect a QD that is dirty, damaged or the wrong type.
• Inspect hoses for kinks, abrasion, swelling or weeping fittings during walks, and report any you find.
• Keep hoses out of rail mechanisms and doors, where they can be pinched.

Working on the cooling system itself

Draining, filling, venting and repairing the TCS or FWS loops is mechanical work done by trained facilities technicians or contractors under a permit and a method of procedure, with lockout of pumps and isolation of valves. Pressurized systems must be depressurized and drained to a known state before fittings are opened, in the same spirit as electrical lockout. Technicians may be asked to help, for example by watching rack temperatures while a loop is switched to its redundant CDU, but they do not open valves, change setpoints or bleed air from the loop themselves.

# Is it safe to open a fitting? Pressure and temperature check before mechanical work (illustrative)
loop = {"pressure": 26, "temperature": 106, "isolated": True, "drained": False}
max_open_pressure = 0.0      # gauge pressure must be zero before a fitting is opened
max_touch_temp = 104
problems = []
if not loop["isolated"]:
    problems.append("isolation valves not closed and locked")
if loop["pressure"] > max_open_pressure:
    problems.append(f"loop still at {loop['pressure']} psi: relieve pressure first")
if not loop["drained"]:
    problems.append("section not drained to the procedure's level")
if loop["temperature"] > max_touch_temp:
    problems.append(f"fluid at {loop['temperature']} F: let it cool or use heat-resistant gloves")
print("Safe to open" if not problems else "NOT safe to open:\n- " + "\n- ".join(problems))

NOT safe to open:
- loop still at 26 psi: relieve pressure first
- section not drained to the procedure's level
- fluid at 106 F: let it cool or use heat-resistant gloves

Fluids and chemicals

Propylene glycol is low in toxicity, which is why it is chosen over ethylene glycol for most data center loops, but concentrated glycol and the treatment chemicals added to it can irritate eyes and skin. Coolant samples are taken regularly by the facilities team or the fluid supplier to check concentration, pH, inhibitors and biological growth. When technicians handle spilled coolant, they wear nitrile gloves and safety glasses, use absorbent materials, and dispose of the used absorbents in the marked containers, because coolant must not go down ordinary drains in large quantities. Immersion cooling fluids, used at some other sites, include hydrocarbon and synthetic oils and engineered fluids; some are combustible, some evaporate, and some need special ventilation. Their safety data sheets govern how they are handled.

Electrical work near liquid

Water and electricity together raise every electrical hazard in lessons 2 to 4. Wet skin lowers body resistance; fluid on equipment creates paths for current and arcs; a puddle under a rack can make the floor part of a circuit. The rules for technicians follow from that: never touch equipment that is wet with coolant, never stand in coolant to work on electrical equipment, and never try to dry or clean equipment that may be energized. Equipment that has been wet is inspected and, if necessary, de-energized by qualified people before anyone else works on it.

Design features a technician should recognize

Liquid-cooled halls include engineering controls that a technician should recognize and respect:

• Drip trays under manifolds and at rack bases, with sensors.
• Manifolds mounted at the rear and below electrical equipment where possible, so a leak runs away from power.
• Isolation valves at each rack, labeled supply and return, so a rack can be cut off without stopping the row.
• Redundant CDUs, so one can be serviced while another carries the load: the same N+1 idea as in electrical systems.
• Pressure relief valves and expansion tanks, which protect the loop from overpressure and must never be blocked.
• Fluid compatible materials and filters, which is why only approved fluids may be added.

The Open Compute Project and ASHRAE's technical committee for data centers publish guidance on liquid cooling, including fluid quality, wetted materials and how technology cooling loops should be designed and operated; Stonebridge's and Corvid AI's procedures are based on that guidance and the equipment makers' instructions.

Walks and inspections

Technicians walk Hall D's galleries and rack rows every shift, and liquid cooling adds items to the walk: dry floors under manifolds and CDUs, no weeping fittings, no kinked or rubbing hoses, drip trays dry, CDU displays showing normal pressure, temperature and flow, no alarms on the leak detection panel, and the leak response kit complete and in place. Anything unusual, even a single drop or a damp patch, is photographed and reported; small leaks rarely stay small, and the cause is easier to find when the leak is new.