Hot and cold aisle containment and airflow management

The single biggest improvement in data centre cooling of the past twenty years cost almost nothing in technology: keeping hot air and cold air apart. Rows of racks are turned to face each other across cold aisles and back to back across hot aisles; then one of the aisles is enclosed so the two airstreams cannot mix. The cooling units can then supply warmer air, return air arrives hotter, and every unit does more work per kilowatt of fan power. But containment works only as well as it is maintained. This lesson covers how containment works, how technicians measure whether it is working, and the daily habits that keep it tight. Blanking panels inside individual racks are covered with rack installation in DCT-130; this lesson looks at the aisle and the hall.

Hot aisles, cold aisles and mixing

In a hall without containment, cold air from the tiles or cooling units fills the cold aisle, enters the server fronts, leaves the backs hot and rises. Some cold air never reaches a server and drifts over the tops of the racks or around their ends to the cooling unit returns. Some hot air drifts the other way, over the tops and around the ends of the rows, into the cold aisle. Two kinds of mixing result:

• Bypass air: cold air that returns to the cooling units without passing through any IT equipment. It is cooling capacity wasted, and it lowers the return temperature, which reduces the cooling units' capacity.
• Recirculation: hot exhaust that finds its way back to server inlets. It raises inlet temperatures, usually at the tops of racks and at the ends of rows, and forces the cooling system to run colder than it otherwise would.

Operators of uncontained halls often compensated by setting the cooling units very cold and moving far more air than the servers needed. Both waste energy. Containment removes the mixing so that the supply air can be warmer and the airflow can match the servers' needs.

Cold-aisle and hot-aisle containment

There are two ways to separate the airstreams:

• Cold-aisle containment: the cold aisle is closed with a roof and doors at each end. Cold air fills the enclosed aisle and can reach only the server inlets; the rest of the room is at the hot exhaust temperature. Hall A was retrofitted with cold-aisle containment in 2019, because the raised floor already delivered air into the cold aisles.
• Hot-aisle containment: the hot aisle is closed, and its hot air is ducted to the cooling unit returns, often through a ceiling plenum or chimney. The rest of the room is at the cold supply temperature. Halls B and C use hot-aisle containment with fan walls.

Each has trade-offs. Cold-aisle containment is easier to retrofit into raised-floor halls but leaves the rest of the room hot, which is less comfortable for staff and can overheat equipment outside the contained rows. Hot-aisle containment keeps the room cool and comfortable but makes the enclosed hot aisle very hot to work in, often 40 °C (104 °F) or more. Work inside a hot aisle follows the heat-stress rules in DCT-120.

Chimney cabinets are a variation of hot-aisle containment for single racks: a duct on the back of the cabinet carries its exhaust directly into the ceiling return.

The parts of a containment system

Jamal walks one of Hall A's contained cold aisles with Roz and lists every part that must be intact:

• End doors: sliding or hinged doors at each end of the aisle, with closers that return them shut and seals at the edges.
• Roof panels: clear or translucent panels over the aisle, often designed to drop away or shrink if a fire suppression system discharges, so the agent can reach the aisle.
• Rack-to-rack gaps: side panels or filler strips closing the spaces between adjacent cabinets.
• Gaps under racks: skirts or seals between the bottom of each cabinet and the floor.
• Gaps above racks: blanking strips between the top of each cabinet and the containment roof, especially where cabinets of different heights stand side by side.
• Empty rack positions: a missing cabinet leaves a large hole in the containment; it must be closed with a blanking frame or a full-height panel.
• Blanking panels inside each rack: the job of the rack's owner and of the DCT-130 install procedures, but checked on every walk.
• Cable openings: brush seals wherever cables pass through the roof or the cabinet tops.

The containment is only as good as its largest hole. An end door propped open for an hour lets enough hot air into a cold aisle to push inlet temperatures up at the end of the row within minutes.

Measuring whether containment is working

Technicians use a few simple numbers to judge airflow management:

• Inlet temperature spread: the difference between the lowest and highest inlet readings in an aisle. In a well-contained aisle, all inlets are within a degree or two of the supply temperature. A large spread means recirculation somewhere.
• Supply-to-inlet rise: how much warmer the rack inlets are than the cooling unit supply air. Ideally less than 2 K (3.6 °F).
• Return-to-exhaust difference: how much cooler the air at the cooling unit return is than the rack exhaust. A large difference means bypass air is diluting the return.
• Pressure across containment: some sites measure the pressure difference between a contained aisle and the room. A slightly positive pressure in a contained cold aisle means the cooling units are supplying a little more air than the servers draw, which is what designers want.

From temperatures alone, a technician can estimate how much of the supply air bypasses the racks and how much of the inlet air is recirculated exhaust. The formulas below are simple mixing balances, assuming air at each point is a blend of the two streams.

# Recirculation and bypass fractions from four temperatures in each aisle
aisles = {  # aisle: (supply, rack inlet, rack exhaust, unit return), C
    "aisle 9": (21.0, 21.8, 34.0, 33.2), "aisle 4": (21.0, 24.6, 33.5, 29.5), "aisle 12": (21.0, 23.4, 34.2, 32.6),
}
print(f"{'aisle':8s} {'recirc':>7s} {'bypass':>7s}")
for name, (sup, inlet, exh, ret) in aisles.items():
    recirc = (inlet - sup) / (exh - sup)
    bypass = (exh - ret) / (exh - sup)
    flag = "  <- investigate" if recirc > 0.10 or bypass > 0.20 else ""
    print(f"{name:8s} {recirc:7.0%} {bypass:7.0%}{flag}")

aisle     recirc  bypass
aisle 9       6%      6%
aisle 4      29%     32%  <- investigate
aisle 12     18%     12%  <- investigate

Aisle 9 is fine. Aisle 4 is still uncontained at one end, waiting for a door, and shows both recirculation and bypass. Aisle 12's numbers changed after a cabinet was removed and its position left open. The script turns an impression into a number Jamal can put in a ticket and track after the fix.

Daily airflow management checks

Every walk through a hall includes airflow management. Roz's list for Hall A:

• End doors closed and closing properly; closers and seals intact. Never prop a door open.
• Roof panels in place, none cracked or lifted.
• No gaps between cabinets, under cabinets or above them.
• Every empty rack slot blanked; every empty rack position closed.
• Cable openings in tops and roofs sealed with brushes.
• No boxes, ladders or carts left inside a contained aisle.
• Perforated tiles only inside contained cold aisles.
• Inlet temperature spread and supply-to-inlet rise on the monitoring screen within the site's targets.

Problems Jamal can fix on the spot under the house rules, such as closing a door, fitting a blanking panel from stock or reseating a brush strip, he fixes and records. Anything else gets a ticket: a broken door closer, a cracked roof panel, a missing side panel on a customer's cabinet.

Customers and containment

In a colocation hall many of the racks belong to customers, and customers' own staff work in them. Containment depends on everyone following the same rules, which are written into Stonebridge's house rules: blank empty slots, close doors, do not remove side panels, and report damage. When a customer leaves a cabinet open or removes blanking, Jamal does not lecture; he fixes what the house rules allow, records it against the cabinet and lets the customer service team follow up. Repeat problems go to the account manager.

At YUL-2, many customers' engineers work in French, and the house rules and containment door signs are in French and English. Jamal learns to explain the rules briefly in both languages, because a sign is easier to ignore than a polite person.

Containment and fire protection

Containment changes how a hall behaves in a fire. Roof panels can block sprinkler spray or prevent a gaseous suppression agent from reaching the aisle. For this reason containment roofs are often made of panels that drop out or shrink at a set temperature, or are linked to the fire alarm so that they open automatically. Smoke detectors may also be needed inside contained aisles. Technicians must never block, tape or wedge these devices, and must report any panel that does not move freely. DCT-120 covers fire suppression safety.

Containment in slab halls

Halls B and C, with hot-aisle containment and fan walls, behave differently from Hall A. The whole room is the cold aisle, at the supply temperature, and the hot aisles are enclosed pods with doors at each end and a ceiling that leads to the return plenum above. A pressure sensor in each pod and in the room lets the fan walls run so that the room stays slightly above the pod pressure. If a pod door is propped open, the pressure difference falls, the fan walls speed up, and hot air leaks into the room near the door. The monitoring system raises an advisory when a pod's pressure difference stays low for more than a few minutes, and the first check is always the doors.

Working inside a hot-aisle pod is hot work in the literal sense. The air can be well above 40 °C (104 °F), and the DCT-120 safety course sets limits on how long a technician may work there without a break and requires water and a buddy for longer jobs. Short tasks such as checking a PDU reading are done quickly with the door closed behind; longer jobs are planned for lower-load periods where possible.

Measuring with a thermal camera

A thermal imaging camera shows temperature as colour, making mixing visible at a glance. Pointed at the front of a row, it shows warm patches at the tops of racks or around gaps where hot air is leaking into a contained cold aisle. Pointed at a containment roof from the hot side, it shows cold patches where cold air is escaping. Stonebridge's technicians carry a small thermal camera on airflow rounds and attach images to tickets. Thermal cameras read surface temperatures, not air temperatures, so they are used to find problems rather than to measure compliance; a calibrated air probe still gives the number that goes in the record.