Heat loads and ASHRAE thermal guidelines

Every watt of electricity delivered to a server becomes a watt of heat. A data hall drawing 20 MW of IT power is, thermodynamically, a 20 MW heater, and the cooling system's job is to carry that heat from the chips, through air or liquid, out of the building and into the atmosphere, a river or a district heating network, continuously and with redundancy, at the lowest practical energy and water cost. Cooling is usually the largest consumer of energy in a data centre after the IT equipment itself, and with AI racks now exceeding 100 kW each, it has become the main constraint on how dense a facility can be.

This course follows the heat along that path. This lesson quantifies heat loads and sets the environmental limits that the IT equipment needs. The section at the end of this lesson maps the rest of the course. The course builds on the thermodynamics and heat transfer of MEC-340.

Where the heat comes from

The heat load of a data hall is the sum of:

• IT equipment: essentially all its electrical input becomes heat in the room, apart from a negligible amount leaving as network signals. This is by far the largest component.
• Power distribution losses inside the space: UPS systems, transformers, PDUs and busway that sit in or near the hall lose a few percent of the power they carry as heat.
• Lighting, people and the building envelope: small in a large hall (lighting perhaps 5 to 10 W/m², a person about 100 W, and solar and conduction gains through walls and roof), but not negligible in small rooms.
• Fans in the cooling equipment itself, whose power is also turned into heat in the airstream.

Designers express load in kilowatts (kW) or megawatts. In North American mechanical practice, cooling capacity is also given in tons of refrigeration and BTU per hour:

A 1 MW hall therefore needs about 284 tons of cooling, before any margin or redundancy.

Density

Load is described per rack and per unit of floor area. Typical enterprise racks drew 3 to 8 kW for many years, and much existing cooling equipment was designed around that. Cloud racks reached 10 to 20 kW, and racks of AI accelerators now range from about 40 kW to well above 100 kW: a single rack of current GPU servers can dissipate 120 kW or more. Density matters because air can only carry so much heat through a rack-sized opening at reasonable speeds and noise levels; beyond roughly 30 to 50 kW per rack, liquid cooling becomes necessary (Lesson 9).

Carrying heat with air and water

Heat is carried by a fluid flow according to the sensible heat equation, \(\dot Q = \dot m c_p \Delta T\). For air at typical conditions (density about 1.2 kg/m³, specific heat 1.006 kJ/kg·K), and for water (1000 kg/m³, 4.19 kJ/kg·K), this becomes:

In inch-pound units the equivalent working formulas are \(\dot Q\,[\text{BTU/h}] = 1.08 \times \text{CFM} \times \Delta T\,[°\text{F}]\) for air and \(500 \times \text{GPM} \times \Delta T\,[°\text{F}]\) for water. The factor of about 3500 between them (per unit volume, water carries 3500 times as much heat as air for the same temperature rise) is the fundamental reason liquid cooling is so much more compact.

The ASHRAE thermal guidelines

IT equipment needs inlet air within certain limits of temperature and humidity. In North America and much of the world, these limits come from ASHRAE Technical Committee 9.9, which publishes the Thermal Guidelines for Data Processing Environments (the 5th edition appeared in 2021). The guidelines define a recommended envelope, within which equipment should operate for best reliability and efficiency, and several allowable classes, within which equipment may operate for limited periods or with some reliability trade-off.

The guidelines also limit moisture. In the recommended envelope the dew point is between −9 °C and 15 °C, with relative humidity at or below 70% (or 50% where gaseous pollutants are present), because high humidity combined with corrosive gases such as sulphur compounds accelerates corrosion of circuit boards. Very dry air raises the risk of electrostatic discharge, but research sponsored by ASHRAE showed the risk to be low with normal grounding practices, which allowed the lower humidity limit to be relaxed and saves the energy once spent humidifying data halls. The temperature must also not change too quickly: the guidelines limit the rate of change to 20 °C in an hour, and to 5 °C in an hour for equipment with tape drives.

Note that these limits apply to the air entering the IT equipment, not to the room average or the return air. A hall can have an average temperature of 24 °C and still have servers ingesting 35 °C air where hot exhaust recirculates; that is why sensors are placed at rack inlets and why airflow management (Lesson 3) matters so much.

Why warmer is better

For decades data halls were kept at 18 to 20 °C, but running warmer has large benefits: chillers become more efficient as the temperature they must produce rises, and the number of hours when outside air or water can do the cooling without compressors (economizer hours, Lesson 7) grows sharply. ASHRAE also published guidance (the "X-factor") estimating how failure rates of volume servers change with inlet temperature. Relative to continuous operation at 20 °C, operating part of the year at higher temperatures raises failure rates modestly, and operating part of the year below 20 °C lowers them, so a facility that runs warm in summer and cool in winter can have nearly the same average failure rate as one held at 20 °C all year.

There are limits. Above about 25 to 27 °C, server fans speed up to keep components cool, and fan power rises with the cube of speed, so total facility energy can rise even as cooling-plant energy falls. Accelerators with very high power often need cooler inlet air or liquid, which is why the H1 class exists. Operators therefore choose a set point by optimizing total energy, not facility energy alone.

Measuring efficiency: PUE and related metrics

Power usage effectiveness (PUE), introduced in DCE-200, is the ratio of total facility energy to IT energy over a year. Cooling is the largest contributor to the overhead above 1.0. A related metric, partial PUE (pPUE), isolates one subsystem: the cooling pPUE is (IT energy + cooling energy)/IT energy. The efficiency of a chilled-water plant is often expressed in kW of electrical input per ton of cooling:

A modern water-cooled chiller plant, including chillers, pumps and towers, may run at 0.5 to 0.7 kW/ton at design conditions and much less when economizing; an older plant may run at 1.0 kW/ton or more.

ASHRAE Standard 90.4, Energy Standard for Data Centers, sets minimum efficiency requirements for the mechanical and electrical systems of data centres, expressed as maximum design annualized mechanical load components that vary with climate zone.

In Canada, energy efficiency of buildings is governed by the National Energy Code of Canada for Buildings (NECB) as adopted by each province (some provinces, such as Ontario and British Columbia, have their own energy requirements and step codes). Data centres are not treated separately in the NECB as they are in ASHRAE 90.4, but the NECB permits compliance through ASHRAE 90.1 in some provinces, and many operators adopt ASHRAE 90.4 targets voluntarily or by owner requirement.

How this course is organized

The course follows the heat from the chip to the outdoors and back to the design table. Lesson 1 covers heat loads and environmental limits, and Lesson 2 the psychrometrics of moist air. Lessons 3 and 4 cover air cooling: airflow management and containment, then the terminal units that cool the air. Lessons 5 and 6 cover chilled-water plants and heat rejection, then pumps, piping and hydronic design. Lesson 7 covers economizers, evaporative cooling and water use, and Lesson 8 controls and monitoring. Lessons 9 and 10 cover liquid cooling, first at the rack and then for whole AI halls. Lesson 11 covers reliability and failure response, and Lesson 12 annual energy, sustainability and a complete design project.