Safety culture in a live critical facility

Jamal Pierce has been a data centre technician at Stonebridge Data for six weeks. He knows the house rules, he can work a remote hands ticket and he has walked every hall with Roz Kerr. This month he starts the safety block that every new technician at the campus completes before working alone: twelve sessions on the hazards of a live critical facility and the habits that keep people and equipment safe. Roz teaches most of them, Dmitri Volkov brings the shift lead's view, and Anjali Desai, the site manager, opens the first one with a sentence Jamal writes down: Nobody at YUL-2 (Nobody at IAD-3) is ever too busy to stop a job.

This course is about safety at technician level. It does not turn a technician into an electrician, a battery specialist or a fire protection engineer. It teaches what a technician must recognize, what they may and may not do, how to protect themselves and the equipment while doing their own work, and when to stop and call someone else. The general workplace safety courses in the safety school, such as electrical safety awareness and lockout, go deeper on each topic; the design side of data centre power, cooling and fire protection belongs to the Data Center Engineering school. Here the focus is the floor of a working building.

Why a data centre is a special workplace

From the visitor corridor, a data hall looks like one of the safest workplaces imaginable: clean, cool, quiet apart from the fans, with no moving vehicles and no open flames. That impression is the first hazard. A data centre concentrates more electrical energy in one building than almost any other workplace. A single hall can draw as much power as a small town, and the switchgear, UPS systems and batteries that supply it can release enormous energy in a fault. The equipment is heavy, it is moved on rolling frames, and the newest AI racks weigh as much as a small car. There are raised floors with open tiles, ladders and lifts for overhead cable trays, hot aisles hotter than a summer afternoon, fans loud enough to damage hearing over a shift, liquid cooling loops under pressure, and fire suppression systems that can discharge gas into the room.

And all of it is live. In most industries, maintenance happens when the plant is shut down. A data centre almost never shuts down. Work is done beside equipment that is running, often a few centimetres from cables and pipes that carry the customers' entire business. That creates two kinds of risk at once: the risk of hurting a person and the risk of interrupting a customer. They are connected. The same carelessness that drops a floor tile on a foot can knock a power cord out of a server, and the same discipline that protects people protects uptime.

Who is responsible for safety

In Quebec, the Act respecting occupational health and safety and its regulations, enforced by the CNESST (Commission des normes, de l'équité, de la santé et de la sécurité du travail), set the duties of employers and workers. The Regulation respecting occupational health and safety (often called the RSST) covers hazards such as noise, ventilation, machines and electrical work in workplaces like a data centre, and the CNESST inspects and investigates. The employer must identify hazards, control them, train workers and supply protective equipment. Workers must follow the rules, use the equipment provided, take care of their own health and safety and that of others, and take part in identifying hazards. Quebec workers also have the right to refuse work they have reasonable grounds to believe is dangerous.

Stonebridge adds its own layer on top of the law: the site safety program, written procedures for hazardous work, a permit-to-work system, and a rule that applies to everyone from a visiting executive to a new technician on their second day: anyone may stop any job they believe is unsafe, without penalty, and the job stays stopped until the concern is resolved. Customers and contractors working on the campus follow the same program. A cloud provider's own technicians in Halls B and C, and Corvid AI's engineers in Hall D, are inducted into Stonebridge's site rules before they receive a badge.

• Stonebridge, as employer and site operator, writes the program, trains its people, controls access and issues permits.
• Contractors, such as electricians, mechanical technicians and fire protection technicians, follow the site program plus their own employers' procedures, under permits issued by Stonebridge.
• Customers' staff follow the house rules in their own cages and may not touch Stonebridge infrastructure.
• Every technician is responsible for doing their own work safely, for not putting others at risk, and for reporting hazards and incidents.

The hierarchy of controls

Every safety program uses the same tool for choosing how to control a hazard: the hierarchy of controls. The most effective controls are at the top and the least effective at the bottom. A good program starts at the top and works down, using the lower controls only for what is left.

• Elimination: remove the hazard entirely. Working on equipment that has been switched off and locked out eliminates the shock hazard for that job.
• Substitution: replace the hazard with something less hazardous. A clean agent that is safe for people at its design concentration substitutes for older agents that were not.
• Engineering controls: isolate people from the hazard. Containment doors, guards on switchgear, finger-safe busway and leak detection under liquid-cooled racks are engineering controls.
• Administrative controls: change the way people work. Permits, procedures, training, signs, escort rules and the two-person rule for high-risk tasks are administrative controls.
• Personal protective equipment: protect the person when the hazard remains. Safety shoes, gloves, hearing protection and arc-rated clothing are the last line of defence, not the first.

In a data centre, the most important control for electrical work is usually elimination: putting the equipment into an electrically safe work condition before anyone touches it. Because the building is designed with redundant A and B paths, one path can usually be switched off and locked out for maintenance while the other carries the load. That is not a coincidence. The concurrent maintainability that the Tier ratings describe is a safety feature as much as a reliability feature.

Two kinds of risk, one discipline

Dmitri's whiteboard in the operations centre shows the redundancy status of every hall: which UPS modules, generators, cooling units and power paths are available and which are out of service. Before any job that touches infrastructure, the shift lead checks the board. A job that would be routine on a normal day may be postponed if one path is already down, because the next mistake would take out the last path. Technicians learn to think about both risks for every task.

Stop work authority

Stop work authority is the right, and the duty, of every person on site to stop a task when they see a hazard that has not been controlled. It does not need a manager's approval and it does not need certainty. A technician who says 'I'm not sure that's safe; let's stop and check' is doing exactly what the program asks. When work is stopped, the people involved make the immediate area safe if they can do so without adding risk, tell the shift lead, and wait. The concern is reviewed, the hazard is controlled or the plan is changed, and the work restarts only when the person who stopped it, or the shift lead, agrees.

Roz tells Jamal about the first time she stopped a job, in her first year. A contractor was about to lift a floor tile beside a row of running racks to pull a cable, without a tile lifter and without cones around the opening, while a delivery cart was coming down the aisle. Stopping him felt awkward. Ten minutes later the contractor had cones, a lifter and a spotter, and the cart had gone around. 'Nobody ever thanks you for an accident that didn't happen,' she says, 'so you have to be the kind of person who stops jobs anyway.'

Reporting hazards, near misses and incidents

A safety program learns only from what it is told. Stonebridge asks every technician to report three kinds of event in the safety reporting system, which is separate from customer tickets:

• Hazards: a condition that could cause harm, such as a missing tile, a damaged cord, a blocked exit, a leaking fitting or a fire door wedged open.
• Near misses: an event that could have caused harm but did not, such as a tile dropped beside someone's foot or a ladder that slipped before anyone climbed it.
• Incidents: any event that caused an injury, damage, a release, or an interruption to a customer.

Near misses matter most, because they are free lessons. For every serious injury there are many near misses with the same causes, and fixing the causes after a near miss prevents the injury. Reports are reviewed weekly by the site safety committee, which includes technicians as well as managers. The aim is to fix conditions and processes, not to blame the person who reported.

# Count safety reports by type and find the most common cause this quarter
reports = [  # (type, cause)
    ("hazard", "open floor tile"), ("near miss", "open floor tile"), ("hazard", "damaged cord"),
    ("hazard", "blocked exit"), ("near miss", "ladder footing"), ("hazard", "open floor tile"),
    ("incident", "pinched finger"), ("near miss", "cart in aisle"), ("hazard", "leaking fitting"),
    ("near miss", "open floor tile"), ("hazard", "door wedged open"), ("near miss", "cart in aisle"),
]
by_type, by_cause = {}, {}
for kind, cause in reports:
    by_type[kind] = by_type.get(kind, 0) + 1
    by_cause[cause] = by_cause.get(cause, 0) + 1
for kind in ("hazard", "near miss", "incident"):
    print(f"{kind:10s} {by_type.get(kind, 0):3d}")
top = max(by_cause, key=by_cause.get)
print(f"Most common cause: {top} ({by_cause[top]} reports)")
ratio = (by_type.get("hazard", 0) + by_type.get("near miss", 0)) / max(1, by_type.get("incident", 0))
print(f"Hazards and near misses per incident: {ratio:.0f}")

hazard       6
near miss    5
incident     1
Most common cause: open floor tile (4 reports)
Hazards and near misses per incident: 11

This quarter's reports point clearly at open floor tiles. The safety committee responds with an engineering control, tile-opening barriers stored at the end of each row, and an administrative one, a rule that no tile is lifted without a barrier in place. Lesson 8 returns to raised floors.

Inductions, training and competence

No one works on the floor at Stonebridge without a site induction covering the emergency procedures, the alarms, the exits, the house rules and the hazards of each space. Technicians then complete training for each task they do, with a mentor signing off when they can do it safely and correctly. Some tasks need recognized training: working near energized electrical equipment requires electrical safety training to the standard the site follows, operating a mobile elevating work platform requires operator training, and entering a battery room or switching power requires the specific authorization of the critical facilities team. Jamal's training record shows exactly which tasks he is authorized to do, and the badge system will not open the switchgear and battery rooms for him until his record shows he is qualified to enter them.

Competence is not the same as experience. Many incidents are caused by experienced people doing a familiar task on autopilot, on a day when something was different. The defences are the same for everyone: read the ticket and the procedure, check the redundancy board, look at the work area before starting, and stop when something does not match.

Emergencies

Every technician must know, from the first day, what to do in an emergency. The induction covers the alarms and their meanings, the exits and assembly points, the location of first aid kits, automated external defibrillators and eyewash stations, and the emergency numbers. In a data centre several alarms are specific: the clean agent pre-discharge alarm, which means leave the room now; the battery room hydrogen or off-gas alarm; the leak detection alarm; and the emergency power off button, which must never be pressed except in a genuine emergency because it cuts power to a whole room. Lesson 11 covers fire alarms and suppression in detail.

If someone is injured, the first step is always to make sure the scene is safe for the rescuer. A person in contact with an energized conductor must not be touched until the power is off. The shift lead coordinates the response, calls emergency services when needed and arranges an escort to meet them at the gate, because emergency crews cannot be left waiting at a secure perimeter.