Hot and cold aisles, blanking and containment at the rack

The cooling of a data hall is designed by mechanical engineers and run by the critical facilities team, and DCT-230 covers it in depth. But most cooling problems a technician meets are made, and fixed, at the cabinet: a missing blanking panel, a gap beside a server, a cable bundle stuffed into a containment door, a cabinet turned the wrong way. Every one of those lets hot exhaust air reach a server's inlet, or lets cold air escape to the hot side without doing any cooling. Roz tells Jamal that a technician with a box of blanking panels and good habits does more for a hall's cooling than any engineer with a new chiller. She shows him with a thermal camera.

Hot aisles and cold aisles

Cabinets in a hall are arranged in rows that face each other in pairs. The fronts of two rows face a shared cold aisle, where cool air is supplied; their backs face hot aisles, where warm exhaust is collected and returned to the cooling units. In Hall A, cool air rises into the cold aisles through perforated tiles in the raised floor. In Halls B and C, cool air is supplied into the room and the hot aisles are enclosed and ducted back to the cooling units. Hall D, with its liquid-cooled racks, still needs air for the heat that the liquid does not carry, about a fifth of its load, and uses the same principle.

The arrangement only works if hot and cold air stay apart. Anywhere they can mix, the cooling system has to work harder, by supplying colder air or more of it, to keep every server inlet cool enough. ASHRAE's widely used guidance recommends server inlet temperatures between about 18 and 27 °C (64 and 81 °F), and most halls run comfortably inside that range as long as the air goes where it should.

Recirculation and bypass

Two kinds of leak spoil airflow, and Roz wants Jamal to be able to name both:

• Recirculation: hot exhaust finding its way back to a server inlet, through an empty unit, around the side of the equipment, over the top of a row or around the end of an aisle. Recirculation causes hot spots, where a server's inlet is much warmer than the supply air.
• Bypass: cold air returning to the cooling units without passing through any equipment, through gaps in the floor, unused cable openings, perforated tiles in the wrong place or gaps around containment. Bypass wastes cooling and fan energy and can starve the cabinets that need the air.

Recirculation is the one that damages equipment; bypass is the one that wastes money. Both are fixed by the same habits at the cabinet.

Blanking panels

A blanking panel is a plain plate, usually plastic or steel, that fills an empty rack unit or units at the front of a cabinet. It seems too simple to matter, and that is why panels are left out. Each empty unit without a panel is a 44 mm (1.75 in) high window through which hot exhaust can flow forward into the cold side of the cabinet, where the nearest servers draw it in. In a cabinet with several empty units, the servers just above and below them can run many degrees warmer than their neighbours.

Stonebridge's rule, which Jamal learned on his first-course tour, is that every empty unit in every cabinet in every hall has a blanking panel, all the time. Tool-less snap-in panels make this easy, and the build room keeps bins of them. When a technician removes equipment, the panels go in before the cabinet is closed. When a technician installs equipment, the removed panels go back into the build room, not onto the floor of the cabinet.

# How much air bypasses the servers through open rack units? (illustrative model)
open_units = 6             # empty units with no blanking panel
gap_area_per_unit = 0.0198       # open area of one empty unit, square metres
pressure_driven_speed = 1.5   # typical speed of air through the gap, m/s
leak = open_units * gap_area_per_unit * pressure_driven_speed * 1000
print(f"{open_units} open units leak about {leak:,.0f} litres per second of air around the servers")
server_air = 45             # air one 1U server moves at full load, litres per second
print(f"That is roughly the air of {leak / server_air:.0f} servers, recirculating hot exhaust instead of cooling")

6 open units leak about 178 litres per second of air around the servers
That is roughly the air of 4 servers, recirculating hot exhaust instead of cooling

The numbers in the code are illustrative, because the actual leak depends on the pressure difference across the cabinet, but the size of the effect is real. Six open units can move as much air as several servers, all of it the wrong way.

Other gaps at the cabinet

Blanking panels close the units; other gaps need other fixes:

• Side gaps between the equipment and the cabinet's side panels, beside the mounting rails, are closed with side air dams or seal kits.
• Gaps under the cabinet, between the floor and the cabinet's bottom, are closed with a bottom panel or skirt.
• Cable openings in the floor or the cabinet top are fitted with brush grommets, which let cables through while closing around them.
• Gaps between neighbouring cabinets in a row are closed by baying or by filler panels.
• Empty cabinet positions in a contained row are closed with blanking doors or a dummy cabinet, so that the containment is not open at that point.

Containment

Containment physically separates the hot and cold air streams. Cold aisle containment encloses the cold aisle with doors at its ends and a roof or panels across its top, so that the only way out for the cold air is through the cabinets. Hot aisle containment encloses the hot aisle and ducts it back to the cooling units, so that the only way in is through the cabinets. Stonebridge uses cold aisle containment in parts of Hall A and hot aisle containment in Halls B and C.

Containment is only as good as its weakest seal, and technicians are often the ones who break it. The most common ways it is broken, in Roz's experience:

• Containment doors propped open, or left open after work.
• A missing cabinet in a contained row, with no blanking door.
• Cables routed through a gap in a containment panel, holding it open.
• Ceiling panels lifted for cable work and not replaced.
• A closer on a containment door that no longer closes it fully, which was one of the findings on Jamal's first-course tour.

Working inside a hot aisle containment is also hot work for people: the air can be well above 35 °C (95 °F). DCT-120 sets the limits on time and the precautions.

Cabling and airflow

Cables affect airflow too. A thick bundle of cables across the rear of a cabinet blocks the server exhausts, raising the temperature inside the cabinet and making fans work harder. Under a raised floor, cable bundles block the air path to the perforated tiles. Good practice keeps cables in the cable managers at the sides of the cabinet, uses cables of the right length so there is no excess to stuff behind the servers, removes abandoned cables, and routes cables overhead in the newer halls, leaving the floor void clear for air.

Checking with your hands and eyes

A technician does not need instruments to spot most airflow problems. Roz's checks, which Jamal now does on every visit:

• Look at the front of every cabinet in the aisle: any empty units without panels? Any equipment facing the wrong way?
• Feel the air at the front of the servers: is it cool everywhere, or warm in places?
• Look along the containment: doors closed? panels in place? any gaps where cables pass?
• Look at the floor: perforated tiles in the cold aisle, not the hot aisle or under a cabinet? cable openings sealed?

Where something seems wrong, a thermal camera or the DCIM temperature sensors on the cabinet fronts confirm it. Stonebridge has sensors on the front of every cabinet at low, middle and high positions, and their readings trend in DCIM. A rising reading at one cabinet's top sensor usually means hot air coming over or around the row, and a rising reading at a middle sensor often means a missing panel.

# Inlet temperature check from cabinet front sensors
recommended = (18, 27)   # ASHRAE recommended inlet range, degrees C
sensors = {  # cabinet: (low, middle, high) front sensor readings, degrees C
    "A-17-02": (20, 21, 22),
    "A-17-03": (20, 22, 23),
    "A-17-04": (21, 27, 29),
    "A-17-05": (20, 21, 23),
}
for cab, (lo, mid, hi) in sensors.items():
    note = []
    if max(lo, mid, hi) > recommended[1]:
        note.append("above recommended range")
    if hi - lo > 5:
        note.append("large spread: look for recirculation over the top or through gaps")
    print(f"{cab}: {lo:.0f} / {mid:.0f} / {hi:.0f}  {'; '.join(note) if note else 'OK'}")

A-17-02: 20 / 21 / 22  OK
A-17-03: 20 / 22 / 23  OK
A-17-04: 21 / 27 / 29  above recommended range; large spread: look for recirculation over the top or through gaps
A-17-05: 20 / 21 / 23  OK

Liquid-cooled racks and air

Hall D's racks take most of their heat away in liquid, but the air side still matters. The roughly one fifth of each rack's heat that goes to air is still many kilowatts per rack, more than a whole Hall A cabinet produces. The power shelves, network switches and storage in an AI rack are usually air-cooled, and so are the parts of the servers that the cold plates do not reach. So Hall D still has hot aisle containment, blanking panels and sealed gaps, and a missing blanking panel in an AI rack causes the same recirculation as one in Hall A, with more heat behind it. Carmen Ortiz's team checks the air side of each rack as part of acceptance, as Lesson 12 shows.

Housekeeping

Airflow also depends on housekeeping. Loose packaging, papers and plastic bags drawn into a cold aisle can block perforated tiles or server inlets. Dust from construction or unpacking clogs server filters and cooling unit coils. Doors left open to corridors let unconditioned air in. Stonebridge's halls are cleaned by specialist contractors on a schedule, using vacuums with fine filters and methods that do not disturb cables, but day-to-day housekeeping is everyone's job: a technician who sees cardboard, a plastic bag or a propped door deals with it or reports it on the spot.