Lockout, isolation and energized work rules

In a factory, lockout usually means stopping a machine, switching off its power and putting a padlock on the disconnect before anyone reaches inside. In a data center the idea is the same, but the setting is different: the equipment being locked out is almost always one half of a redundant system whose other half is carrying a live customer load. Getting the isolation wrong can injure the person working, and it can also switch off the wrong half and drop the load. This lesson explains how lockout works at Stonebridge, what a technician's part in it is, and the rules for the rare cases where work is done on energized equipment.

The general principles of hazardous energy control are taught in the safety school's lockout course. Here they are applied to the data center, where a technician most often meets lockout as an affected worker, a second person checking the procedure, or the person locking out their own small task, such as a rack power strip being replaced by a qualified electrician.

What lockout controls

Hazardous energy is any energy that could be released unexpectedly and hurt someone. In a data center it includes:

• Electrical energy from the utility, the generators, the UPS output and the batteries, which stay energized even when the utility is off.
• Stored electrical energy in capacitors inside UPS modules, variable-frequency drives and power supplies.
• Mechanical energy in fans, pumps and rotating parts of cooling units.
• Pressure in chilled water, condenser water and liquid cooling loops, and in fire suppression cylinders and pre-action piping.
• Thermal energy in hot pipes and in equipment that has just been running.
• Gravity, in raised equipment, lift tables and doors held open.

A piece of equipment may have several sources at once. A UPS module, for example, has an AC input, a DC battery input, a bypass input and an AC output that may be back-fed from a parallel module, plus capacitors inside. Locking out only the input breaker leaves it lethal.

The steps of a lockout

Stonebridge's lockout program follows OSHA's control of hazardous energy standard, 29 CFR 1910.147, and the electrical safe work practices in 1910.333, as adopted by VOSH, together with NFPA 70E for electrical work. Every piece of infrastructure has a written, equipment-specific lockout procedure that lists each energy source, each isolation point, the device used to lock it and the method of verification.

Whatever the equipment, a lockout follows the same sequence:

• Prepare: read the procedure and the method of procedure, identify every energy source and isolation point, check the redundancy board and notify affected people, including customers if the change process requires it.
• Shut down: stop the equipment using its normal controls, after transferring its load to the redundant path.
• Isolate: open the disconnects, breakers or valves at every isolation point. For electrical equipment, a qualified person does this.
• Lock and tag: each worker puts their own personal lock and tag on each isolation point, or on a group lockbox that holds the keys. Nobody uses someone else's lock.
• Release stored energy: discharge capacitors, bleed pressure, let rotating parts stop, block or lower raised parts.
• Verify: try the normal start controls to confirm nothing starts, then a qualified person tests for absence of voltage at the point of work with a tested meter.
• Do the work.
• Restore: remove tools, refit guards, account for every person, remove locks one by one, re-energize under the procedure and return the load.

Testing for absence of voltage

The most important step is verification. A breaker can be labeled wrongly, a drawing can be out of date, a back-feed can come from somewhere unexpected. The only proof that equipment is dead is a test at the point of work. Qualified workers use the live-dead-live method: test the meter on a known live source to prove it works, test the de-energized conductors phase to phase and phase to ground, then test the meter again on the known source to prove it did not fail during the test. Some newer switchgear has permanently installed absence of voltage testers, which do the test automatically and show the result on an indicator. Technicians do not perform these tests, but they should know what they look like and refuse to start their own part of a job until they are told the test has been done.

# Check a live-dead-live verification record before work starts
record = [  # (step, reading in volts)
    ("meter on known source, before", 481.0),
    ("L1-L2", 0.0), ("L2-L3", 0.0), ("L1-L3", 0.4),
    ("L1-ground", 0.0), ("L2-ground", 0.0), ("L3-ground", 0.0),
    ("meter on known source, after", 481.0),
]
threshold = 3.0    # readings below this are treated as absence of voltage (program value, illustrative)
before, after = record[0][1], record[-1][1]
ok_meter = before > 50 and after > 50
dead = all(v < threshold for _, v in record[1:-1])
for step, v in record:
    print(f"{step:32s} {v:7.1f} V")
print("Meter proven before and after:", "yes" if ok_meter else "NO - repeat the test")
print("Absence of voltage confirmed:", "yes" if (dead and ok_meter) else "NO - do not start work")

meter on known source, before      481.0 V
L1-L2                                0.0 V
L2-L3                                0.0 V
L1-L3                                0.4 V
L1-ground                            0.0 V
L2-ground                            0.0 V
L3-ground                            0.0 V
meter on known source, after       481.0 V
Meter proven before and after: yes
Absence of voltage confirmed: yes

Group lockout

Large jobs, such as a UPS module replacement or switchgear maintenance, involve several workers from several trades. Stonebridge uses group lockout for them. The person in charge, usually a qualified member of the critical facilities team, isolates and locks every point with the job's locks and puts the keys in a group lockbox. Each worker then puts their own personal lock on the lockbox. Nobody can open the lockbox, and so nobody can remove the job locks, until every worker has removed their own lock, which they do only when their part of the work is finished and they are clear. A technician working inside the locked-out area puts their own lock on the box too; it is the only thing that guarantees the equipment cannot be re-energized while they are there.

# Group lockbox: can the job locks be removed yet?
lockbox = {"locks on box": ["E. Tremblay (electrician)", "J. Pierce (technician)", "K. Osei (UPS vendor)"]}
finished = {"E. Tremblay (electrician)": True, "J. Pierce (technician)": True, "K. Osei (UPS vendor)": False}
still_on = [p for p in lockbox["locks on box"] if not finished[p]]
for p in lockbox["locks on box"]:
    print(f"{p:28s} {'removed own lock' if finished[p] else 'LOCK STILL ON - still working'}")
print("Job locks may be removed" if not still_on else f"Job locks stay on: {len(still_on)} worker(s) still protected")

E. Tremblay (electrician)    removed own lock
J. Pierce (technician)       removed own lock
K. Osei (UPS vendor)         LOCK STILL ON - still working
Job locks stay on: 1 worker(s) still protected

Lockout in a redundant system

In a 2N system, the A and B paths are separate all the way from the switchgear to the rack. Locking out part of the B path means every rack it feeds runs on A alone for the duration of the job. That changes the risk picture for everyone in the building:

• The A path is now the only path. Any work that could disturb it, even a cable pull near an A-side power strip, is postponed or done with extra care.
• Single-corded equipment on the B side, or on a transfer switch, may lose power; the change process must find it beforehand.
• Customers are notified that their B feed will be off, and asked to confirm that their equipment is correctly dual-corded.
• The shift lead marks the redundancy board, and monitoring alarms for the B path are suppressed only as the method of procedure allows.

Isolation points in a data center are labeled with the equipment name and path, such as UPS-B2 OUTPUT or PDU-A-07 MAIN. A common and dangerous error is isolating the right equipment on the wrong path: opening an A breaker when the procedure says B. Stonebridge's procedures require the person isolating to read the label aloud and the second person to confirm it against the procedure before the breaker is touched.

Rack-level isolation

Some isolation happens at the rack. When a power strip in a customer cabinet must be replaced, the qualified electrician switches off the strip's branch circuit at the PDU or busway tap-off, locks and tags it, and verifies absence of voltage at the strip before unplugging it. The technician's job is usually to check, with the customer, that every device on that strip has its other cord on the other strip and is running on it. A device that is single-corded or has a failed second power supply will go down when the strip is switched off, so it is found and dealt with first.

# Before a B strip is switched off: which devices would lose power?
devices = [  # (name, cord on A strip, cord on B strip, A supply healthy)
    ("web-11", True, True, True), ("web-12", True, True, True), ("db-04", True, True, False),
    ("kvm-01", False, True, True), ("fw-02", True, True, True),
]
at_risk = []
for name, a, b, a_ok in devices:
    if not b:
        continue
    if not a:
        at_risk.append((name, "single-corded on B"))
    elif not a_ok:
        at_risk.append((name, "A power supply failed"))
for name, why in at_risk:
    print(f"STOP: {name} would go down ({why})")
if not at_risk:
    print("All devices on the B strip will stay up on A")

STOP: db-04 would go down (A power supply failed)
STOP: kvm-01 would go down (single-corded on B)

Here two devices would go down. The customer replaces the failed power supply in db-04 and moves kvm-01 to an automatic transfer switch before the strip is switched off.

Energized work

Sometimes work must be done on equipment that is energized. The standards allow it only when de-energizing would create a greater hazard, such as switching off life-safety equipment, or is infeasible because of equipment design or the nature of the task, such as voltage testing itself. Convenience and the wish to avoid scheduling an outage are not justifications. When energized work is justified, it requires:

• An energized electrical work permit describing the task, the justification, the shock and arc flash risk assessment, the protective equipment, the boundaries, the people involved and the emergency plan, approved by a manager.
• Qualified persons only, with the shock and arc flash PPE the assessment calls for.
• A safety watch who can switch off the power and call for help.
• Barriers at the boundaries and a job briefing before work starts.

Diagnostic work such as voltage measurement, thermography and checking indications may be done energized by qualified persons under the program without a separate permit, but still with the right PPE. Technicians reading the displays on closed equipment, or the screen of a power strip, are not doing energized work; opening a cover is.

Locks, tags and their care

Personal locks at Stonebridge are red, keyed individually, and labeled with the worker's name and photo. Each worker has one key, which stays with them. Tags carry the worker's name, the date, the reason and a contact number. A tag alone is not a lock: it warns but does not prevent. Tags are used without locks only where a lock physically cannot be attached and the program adds an extra measure, such as removing a fuse. If a worker leaves site with their lock still on, the lock is removed only under a formal procedure: the supervisor confirms the worker is not on site and has been contacted, the equipment is checked as safe, and the removal is recorded and the worker told before they return.