Outside plant, diverse routes and carrier entrances

A data center is only as reliable as its connections to the outside world. The fibers that join it to carriers, internet exchanges, other data centers on the campus and the operator's other regions run through the outside plant (OSP): duct banks, maintenance holes, poles, bridges and road crossings that the operator often does not own and cannot fully see. Most serious connectivity outages are caused not by equipment but by cable cuts, typically by excavation, and the defense against them is physical diversity: more than one route, entering the building at more than one point, with nothing shared between them.

This lesson covers OSP construction, high-count fiber cables and splicing, long-link loss budgets, how to design and verify diverse routes, building entrance facilities and the code rules that apply to them, and the commercial arrangements for carrier connectivity.

Outside plant construction

Fiber between buildings and to carriers is placed in one of three ways:

• Underground in duct banks: groups of conduits, typically 100 mm (4 in) in diameter, laid in a trench and often encased in concrete, with maintenance holes (manholes) or handholes at intervals for pulling, splicing and storing slack. This is the standard for campuses and for the routes immediately around a data center.
• Directly buried: armored cable placed in a trench or plowed in without conduit. Cheaper, but the cable cannot be replaced or added to without digging again.
• Aerial: cable lashed to a steel strand on utility poles. Quick and cheap where poles exist, but exposed to storms, vehicle strikes, ice and fire.

Inside each conduit, innerduct (smaller plastic ducts, often three per 100 mm conduit) or fabric innerduct (flexible textile sleeves that pack more cables into a conduit) subdivides the space, so that each new cable can be pulled without disturbing the others. Microduct bundles allow fiber cables to be blown in with compressed air, a method called air-blown fiber or jetting, which places cable over long distances with little pulling tension and allows capacity to be added later.

Trenchless methods avoid open excavation where it is impractical. Horizontal directional drilling (HDD) bores under roads, railways and rivers by steering a drill head along a planned path and pulling conduit back through the hole. Micro-trenching cuts a narrow slot in the road surface for microduct, which is fast but shallow and therefore easier to damage later.

Maintenance holes are placed at changes of direction, at splice points and at intervals set by the cable's allowable pulling tension and by the number of bends; for fiber pulled in conduit, spacing is commonly a few hundred meters, and longer where cable is blown in. At each maintenance hole, a slack loop (commonly 15 to 30 m) is stored on racks so that the cable can be pulled out for splicing or repairs without adding a new splice.

OSP depth and routing are set by the owner's standards and by the authority granting the right of way. Telecommunications ducts are commonly buried with about 30 to 42 in of cover. Before any excavation, the contractor must call or submit a request to 811, the national call-before-you-dig number, which routes it to the state one-call center so that owners of buried facilities can mark them.

High fiber-count cables and splicing

Campus and inter-building cables have grown enormously with AI clusters and multi-building campuses. Cables of 864, 1728 and 3456 fibers are common, and 6912-fiber cables are used between large buildings. These cables use ribbon fiber: 12 fibers bonded side by side, so that a whole ribbon can be spliced in one operation with a mass fusion splicer. Rollable ribbon, in which the fibers are joined only at intermittent points, can be rolled into a compact bundle, putting many more fibers into a given duct than flat ribbons.

Splices are housed in sealed splice closures in maintenance holes or vaults, organized in trays with each fiber identified by its color code. Standard fiber colors repeat in a 12-color sequence: blue, orange, green, brown, slate, white, red, black, yellow, violet, rose and aqua, with tubes or ribbons identified by the same sequence.

Long-link loss budgets

Loss budgets for OSP links follow the same method as in Lesson 6, but the fiber attenuation and splices become significant, and an allowance is added for future repairs: each time a cable is cut and repaired, at least one or two splices are added.

Route diversity

Two routes are diverse only if no single event can cut both. Shared risks are often hidden:

• Two carriers may lease fiber in the same duct bank, or in the same conduit of a third party, for part of their routes.
• Different routes often converge at bridges, railway and highway corridors, river crossings and the last few hundred meters before a building.
• Two "diverse" entrances may join inside the building in the same entrance room or the same tray.
• Two carriers may both rely on the same upstream exchange, central office or power supply.

Network engineers call each such shared element a shared risk link group (SRLG). The designer verifies diversity by obtaining route maps (usually as geographic files) from each carrier, overlaying them, and identifying every point where the routes come within a set distance of each other, often a few tens of meters. The contracts should then commit the carriers to maintaining the separation and to notifying the operator before any re-route. Diversity is not permanent: carriers re-route fiber during road works and network changes, so it must be re-checked periodically.

TIA-942 calls for diverse entrance pathways for higher rating levels, with maintenance holes on separate sides of the building and recommends that they be separated by at least 20 m. Within the building, each entrance should lead to its own entrance room or to a separate area, and backbone cabling from each to the MDAs should follow separate pathways.

Building entrance facilities

Where the OSP cables enter the building, the design must keep out water and gas, protect the cables and the people working on them, and meet code rules for outside plant cable inside a building:

• Entrance conduits slope down away from the building so water drains outward, and every conduit, including spares, is sealed with duct plugs or an expanding sealant.
• OSP cables are designed for water resistance and strength, not for fire performance, and are usually not listed for indoor use. They are therefore spliced to listed indoor cable, or enclosed in a suitable raceway, close to the entrance point.
• Metallic members in OSP cables (armor, strength members) must be bonded to the building's telecommunications bonding system at the entrance, and copper circuits from outside need primary protectors that limit lightning and power-cross voltages.

In the US, the NEC covers these entrance requirements: unlisted outside plant optical fiber cable may extend into the building no more than 15 m (50 ft) from the point of entrance before terminating in an enclosure, unless it is in a suitable raceway (770.48); conductive members must be bonded or interrupted at the entrance (770.93 and 770.100); and communications circuits that may be exposed to lightning or power contact need listed primary protectors (800.90).

The entrance room then holds the splice enclosures or OSP termination panels, carrier equipment and the patch panels that connect to the building backbone. In multi-tenant and colocation buildings, carriers often have their own locked cages or cabinets in the entrance room, and the operator's staff make the cross-connects between them.

Carrier connectivity

A data center connects to the outside world through several kinds of service and arrangement:

• Lit services: a carrier provides bandwidth, such as an Ethernet circuit or a wavelength at a given speed, over its own equipment. The carrier manages the optics and the network.
• Dark fiber: the operator leases fibers with no equipment and lights them with its own optics, gaining capacity limited only by its equipment. Long-term dark fiber is often acquired as an indefeasible right of use (IRU), a prepaid lease of typically 15 to 25 years.
• Internet exchange points and peering: the operator connects to networks at an exchange or in a carrier hotel (a building where many carriers interconnect) to exchange traffic directly (DCE-210).
• Cross-connects in colocation: in a colocation facility, the customer pays the operator a monthly fee for each cross-connect from its cage to a carrier in the meet-me room. A carrier-neutral facility, with many competing carriers present, is valuable to tenants.

Building routes in public rights of way requires permits or franchise agreements from the city, county or state transportation department that controls the road, and railway crossings need agreements with the railroad. Attachments to utility poles are governed by agreements with the pole owner and by FCC pole attachment rules (47 U.S.C. § 224), or by the state in states that regulate attachments themselves. Data center developers who are not carriers usually contract with a carrier or a fiber builder to construct and own the off-site routes.

The schedule for OSP is often longer than for the building cabling: permits, road crossings and a single uncooperative landowner can add months. As noted in DCE-200, diverse fiber routes into a site must be planned with the site selection, not after the building is designed.

Lesson 12 draws the course together in a complete project, including the diverse entrances and routes covered here.