Outside plant, diverse routes and carrier entrances

A data centre is only as reliable as its connections to the outside world. The fibres that join it to carriers, internet exchanges, other data centres on the campus and the operator's other regions run through the outside plant (OSP): duct banks, maintenance holes, poles, bridges and road crossings that the operator often does not own and cannot fully see. Most serious connectivity outages are caused not by equipment but by cable cuts, typically by excavation, and the defence against them is physical diversity: more than one route, entering the building at more than one point, with nothing shared between them.

This lesson covers OSP construction, high-count fibre cables and splicing, long-link loss budgets, how to design and verify diverse routes, building entrance facilities and the code rules that apply to them, and the commercial arrangements for carrier connectivity.

Outside plant construction

Fibre between buildings and to carriers is placed in one of three ways:

• Underground in duct banks: groups of conduits, typically 100 mm (4 in) in diameter, laid in a trench and often encased in concrete, with maintenance holes (manholes) or handholes at intervals for pulling, splicing and storing slack. This is the standard for campuses and for the routes immediately around a data centre.
• Directly buried: armoured cable placed in a trench or ploughed in without conduit. Cheaper, but the cable cannot be replaced or added to without digging again.
• Aerial: cable lashed to a steel strand on utility poles. Quick and cheap where poles exist, but exposed to storms, vehicle strikes, ice and fire.

Inside each conduit, innerduct (smaller plastic ducts, often three per 100 mm conduit) or fabric innerduct (flexible textile sleeves that pack more cables into a conduit) subdivides the space, so that each new cable can be pulled without disturbing the others. Microduct bundles allow fibre cables to be blown in with compressed air, a method called air-blown fibre or jetting, which places cable over long distances with little pulling tension and allows capacity to be added later.

Trenchless methods avoid open excavation where it is impractical. Horizontal directional drilling (HDD) bores under roads, railways and rivers by steering a drill head along a planned path and pulling conduit back through the hole. Micro-trenching cuts a narrow slot in the road surface for microduct, which is fast but shallow and therefore easier to damage later.

Maintenance holes are placed at changes of direction, at splice points and at intervals set by the cable's allowable pulling tension and by the number of bends; for fibre pulled in conduit, spacing is commonly a few hundred metres, and longer where cable is blown in. At each maintenance hole, a slack loop (commonly 15 to 30 m) is stored on racks so that the cable can be pulled out for splicing or repairs without adding a new splice.

OSP depth and routing are set by the owner's standards and by the authority granting the right of way. Telecommunications ducts are commonly buried with about 750 mm to 1 m of cover. Before any excavation, the contractor must request locates from the provincial one-call service (such as Ontario One Call, Info-Excavation in Quebec or BC 1 Call), which notifies the owners of buried facilities so they can mark them.

High fibre-count cables and splicing

Campus and inter-building cables have grown enormously with AI clusters and multi-building campuses. Cables of 864, 1728 and 3456 fibres are common, and 6912-fibre cables are used between large buildings. These cables use ribbon fibre: 12 fibres bonded side by side, so that a whole ribbon can be spliced in one operation with a mass fusion splicer. Rollable ribbon, in which the fibres are joined only at intermittent points, can be rolled into a compact bundle, putting many more fibres into a given duct than flat ribbons.

Splices are housed in sealed splice closures in maintenance holes or vaults, organized in trays with each fibre identified by its colour code. Standard fibre colours repeat in a 12-colour sequence: blue, orange, green, brown, slate, white, red, black, yellow, violet, rose and aqua, with tubes or ribbons identified by the same sequence.

Long-link loss budgets

Loss budgets for OSP links follow the same method as in Lesson 6, but the fibre attenuation and splices become significant, and an allowance is added for future repairs: each time a cable is cut and repaired, at least one or two splices are added.

Route diversity

Two routes are diverse only if no single event can cut both. Shared risks are often hidden:

• Two carriers may lease fibre in the same duct bank, or in the same conduit of a third party, for part of their routes.
• Different routes often converge at bridges, railway and highway corridors, river crossings and the last few hundred metres before a building.
• Two "diverse" entrances may join inside the building in the same entrance room or the same tray.
• Two carriers may both rely on the same upstream exchange, central office or power supply.

Network engineers call each such shared element a shared risk link group (SRLG). The designer verifies diversity by obtaining route maps (usually as geographic files) from each carrier, overlaying them, and identifying every point where the routes come within a set distance of each other, often a few tens of metres. The contracts should then commit the carriers to maintaining the separation and to notifying the operator before any re-route. Diversity is not permanent: carriers re-route fibre during road works and network changes, so it must be re-checked periodically.

TIA-942 calls for diverse entrance pathways for higher rating levels, with maintenance holes on separate sides of the building and recommends that they be separated by at least 20 m. Within the building, each entrance should lead to its own entrance room or to a separate area, and backbone cabling from each to the MDAs should follow separate pathways.

Building entrance facilities

Where the OSP cables enter the building, the design must keep out water and gas, protect the cables and the people working on them, and meet code rules for outside plant cable inside a building:

• Entrance conduits slope down away from the building so water drains outward, and every conduit, including spares, is sealed with duct plugs or an expanding sealant.
• OSP cables are designed for water resistance and strength, not for fire performance, and are usually not listed for indoor use. They are therefore spliced to listed indoor cable, or enclosed in a suitable raceway, close to the entrance point.
• Metallic members in OSP cables (armour, strength members) must be bonded to the building's telecommunications bonding system at the entrance, and copper circuits from outside need primary protectors that limit lightning and power-cross voltages.

The Canadian Electrical Code (Sections 56 and 60) sets these entrance requirements in Canada, including bonding of conductive members and protection of communication circuits that enter the building, and limits how far outside plant cable may run inside a building before it must be terminated or enclosed in a suitable raceway.

The entrance room then holds the splice enclosures or OSP termination panels, carrier equipment and the patch panels that connect to the building backbone. In multi-tenant and colocation buildings, carriers often have their own locked cages or cabinets in the entrance room, and the operator's staff make the cross-connects between them.

Carrier connectivity

A data centre connects to the outside world through several kinds of service and arrangement:

• Lit services: a carrier provides bandwidth, such as an Ethernet circuit or a wavelength at a given speed, over its own equipment. The carrier manages the optics and the network.
• Dark fibre: the operator leases fibres with no equipment and lights them with its own optics, gaining capacity limited only by its equipment. Long-term dark fibre is often acquired as an indefeasible right of use (IRU), a prepaid lease of typically 15 to 25 years.
• Internet exchange points and peering: the operator connects to networks at an exchange or in a carrier hotel (a building where many carriers interconnect) to exchange traffic directly (DCE-210).
• Cross-connects in colocation: in a colocation facility, the customer pays the operator a monthly fee for each cross-connect from its cage to a carrier in the meet-me room. A carrier-neutral facility, with many competing carriers present, is valuable to tenants.

Building routes in public rights of way requires consent from the road authority. In Canada, telecommunications carriers have rights under the federal Telecommunications Act to construct on highways and public places with the consent of the municipality, and disputes over the terms can be taken to the CRTC. Attachments to utility poles are governed by agreements with the pole owner, and, depending on the owner, by provincial regulators or the CRTC. Data centre developers who are not carriers usually contract with a carrier or a fibre builder to construct and own the off-site routes.

The schedule for OSP is often longer than for the building cabling: permits, road crossings and a single uncooperative landowner can add months. As noted in DCE-200, diverse fibre routes into a site must be planned with the site selection, not after the building is designed.

Lesson 12 draws the course together in a complete project, including the diverse entrances and routes covered here.