Pathways, cable trays and fill

Every cable in a data center needs a route: a pathway that supports it, protects it, keeps it away from things that would damage or interfere with it, and leaves room for the cables that will be added later. Pathways are also a large part of the coordination problem in a data hall. Cable trays share the ceiling space with power busway, lighting, fire sprinkler pipes, containment structures and sometimes liquid cooling pipework, and each has its own clearances. A pathway plan drawn early, with heights and routes for each system, saves expensive conflicts during construction.

This lesson covers the types of pathway, how to lay them out in a hall, how to calculate fill and weight, bend radius and pulling limits, separation from power, fire ratings of cables, firestopping, and the bonding of pathways and cabinets.

Types of pathway

• Ladder rack: two side rails joined by rungs, usually steel, supporting heavy cable bundles. Common in network rooms and above rows in older designs.
• Wire mesh (basket) tray: a light welded-wire tray that can be cut and bent on site for drops and changes of direction. The most common pathway above cabinet rows in modern halls.
• Solid-bottom and ventilated trays: used where cables need more protection or support, such as in risers.
• Fiber raceway: plastic duct systems (often yellow) with smooth, controlled-radius fittings, made for patch cords and small fiber cables that would be crushed or bent too tightly in wire tray.
• Conduit: enclosed metal or PVC raceway for routes through walls, between buildings, in areas needing physical protection, and wherever the code or building design requires it.
• Under-floor pathways: trays or baskets under a raised access floor. Once standard, they are now used less for cabling, because cables under the floor block the cooling air that the floor plenum carries, are hard to trace, and accumulate abandoned cable. Many raised-floor halls run power under the floor and data overhead, or put everything overhead.

TIA-942 recommends that overhead pathways be arranged in layers by system, for example power at one level, copper data above or below it and fiber raceway at another, so that each can be accessed and expanded without disturbing the others, and so that power and data cross at right angles where they must cross.

Laying out pathways

A good pathway layout follows a few principles:

• Follow the rows. The main pathways run along the rows, above the cabinets, with cross-aisle pathways connecting them to the HDA and MDA. Drops into each cabinet come from directly above.
• Keep the hot and cold aisles clear. Pathways above cabinets rather than above the aisles leave space for containment panels and lighting, and avoid blocking airflow. Where trays must cross a contained aisle, the containment roof must be designed around them.
• Leave clearance. Trays need at least about 300 mm of clear space above them so that cables can be placed by hand, and must keep the clearance that fire codes require below sprinkler heads (typically 450 mm, depending on the sprinkler standard).
• Provide diverse routes. For redundant designs, the A and B paths from each HDA to the MDA should run on separate pathways, ideally on opposite sides of the hall, so that a single incident cannot cut both.
• Plan for growth. Trays are sized for the cabling expected at full build-out plus margin, because adding tray above a live, full hall is difficult and risky.

Tray fill

Cables are not placed in a tray in neat rows, so the usable space is less than the tray's cross-sectional area. TIA-569 limits tray fill to 50% of the inside cross-sectional area and the depth of cable to 150 mm, and good practice is to design for no more than about 25% at initial installation so that the tray can absorb future growth. The fill is calculated from the cables' cross-sectional areas:

where \(N_i\) is the number of cables of diameter \(d_i\), and \(W\) and \(D\) are the inside width and usable depth of the tray.

Weight and support

Trays must also carry the cables' weight. Each tray has a load rating, in kg/m (or lb/ft), for a given span between supports, and the supports must carry the load to the structure. A Category 6A cable weighs roughly 0.06 to 0.08 kg/m, so a few hundred of them weigh more than the tray itself.

Bend radius and pulling tension

Cables are damaged by bending them too tightly or pulling them too hard. Over-bent copper loses its pair geometry and fails crosstalk and return loss tests; over-bent fiber loses light (macrobending loss) and can crack. The standards and manufacturers set limits:

• Category 6A copper: minimum installed bend radius about 4 times the cable diameter; maximum pulling tension about 110 N (25 lbf).
• Fiber cable: minimum bend radius 10 times the cable diameter when installed and not under load, and 15 to 20 times while being pulled under tension (TIA-568.3 and manufacturer data). Bend-insensitive fibers tolerate tighter bends in the fiber itself, but the cable's structure still sets the limit.
• Pulling tension for fiber cable is set by the manufacturer from its strength members, often 600 to 2700 N for backbone cables, and must not be exceeded; long pulls use tension-limiting swivels or are measured with a dynamometer.

Tray fittings (bends, tees and waterfalls at drops) and fiber raceway fittings are made with large radii for this reason. Cable ties pulled tight crush cables and deform pairs; hook-and-loop straps, applied loosely, are the standard for data cabling.

Conduit for telecommunications

Conduits for telecommunications cables follow stricter rules than those for power. TIA-569 recommends:

• No more than two 90° bends (180° in total) between pull points, compared with 360° for electrical raceway.
• No more than 30 m (100 ft) between pull points.
• Fill of no more than 40% of the conduit area for three or more cables (the same figure used for power conductors), and less when long pulls or bends demand it.
• A bend radius of at least 6 times the conduit diameter for conduits up to 50 mm, and 10 times for larger conduits.

Pull boxes must be sized and placed so the cable is not bent tighter than its own limit inside them. A pull cord left in each conduit, and spare conduits, make future cables much cheaper to add.

Separation from power

Power cables can induce noise in unshielded twisted-pair copper. TIA-942 and TIA-569 give minimum separation distances that grow with the number and current of the power circuits, and shrink when the power cables are in grounded metal conduit or when the data cables are shielded. In practice most data centers simply give power and data their own pathways at different levels, crossing at right angles. Fiber is immune to electrical interference, but is still kept in separate pathways so that power work does not disturb it.

The electrical code also sets safety separations. The NEC (800.133 for communications wires and 770.133 for optical fiber cables) does not allow communications cables in the same raceway, cable tray compartment or enclosure as power conductors unless they are separated by a permanent barrier or the rules otherwise permit it, and requires minimum spacing from power conductors.

Fire ratings of cables

Cables carry a large amount of combustible insulation and jacket material, and in a fire they can spread flame and smoke along the pathway. The building and electrical codes therefore require cables to be listed for the space in which they run.

In the US, the NEC lists cable types by fire resistance, with the most resistant permitted to replace the less resistant. For communications cable: CMP (plenum), CMR (riser), CMG and CM (general purpose), and CMX (limited use). For optical fiber cable (Article 770): OFNP and OFCP (plenum), OFNR and OFCR (riser), and OFNG, OFN, OFCG and OFC (general purpose), where N means nonconductive and C conductive (having metallic members). Plenum-rated cable is required in spaces used for environmental air; riser cable in vertical runs between floors. Article 645 allows certain relaxations under the raised floor of an information technology equipment room that meets its conditions.

A raised floor used as an air supply plenum, or a ceiling return plenum, can trigger the plenum rating requirement. Because the price difference is now small, many designers specify plenum-rated cable throughout a data center so that any cable can be moved to any space.

The NEC also requires the accessible portion of abandoned cables (those not terminated at both ends and not tagged for future use) to be removed (770.25 and 800.25). Abandoned cable adds fire load, blocks pathways and airflow, and hides the live cables.

Firestopping

Wherever a pathway passes through a fire-rated wall or floor, the opening must be sealed with a tested firestop system so that the wall keeps its fire-resistance rating. In the US, firestop systems are tested to ASTM E814 or UL 1479 and must match a listed design for that wall type, penetrating items and opening size. Data centers use re-enterable firestop devices (pathway sleeves with built-in intumescent seals, or pillows and putty) so that cables can be added without destroying the seal.

Bonding of pathways and cabinets

Metal pathways, racks and cabinets must be bonded so that a fault on any power circuit cannot leave them energized, and so that the whole data hall sits at a common reference potential, which helps control noise on shielded cabling. TIA-607 defines a telecommunications bonding system: a primary bonding busbar (PBB) in the entrance room or main electrical room connected to the building's grounding electrode system, secondary bonding busbars (SBB) in each telecommunications space, and a telecommunications bonding backbone conductor connecting them.

In the computer room, a mesh common bonding network (mesh-BN) is commonly installed: a grid of bare copper conductors, typically on a 600 mm to 3 m spacing, under the raised floor or above the cabinets, bonded to the building steel and the SBB. Each cabinet and rack is bonded to the mesh with its own conductor (often 6 AWG copper) and listed compression connectors, and cable trays are bonded along their length and across every joint where the tray is not listed as electrically continuous. Bonds are made with a direct, short conductor; daisy-chaining cabinets together is not acceptable because removing one cabinet would break the bond to the rest.

Lesson 9 applies these pathway principles to the extreme fiber volumes of AI clusters, and Lesson 10 continues inside the cabinet with cable managers, airflow and administration.