Inspection, testing, maintenance and impairments

Fire protection systems spend almost their whole lives waiting. A sprinkler valve may sit closed for twenty years, a detector may never see smoke, and a clean agent cylinder may never discharge. Unlike the power and cooling systems of a data centre, which fail visibly when they stop working, a standby fire protection system can fail silently and stay failed until the moment it is needed. The only way to know that it will work is to inspect, test and maintain it on a schedule, to manage carefully the periods when it must be taken out of service, and to control the activities, such as hot work and construction, that most often start fires. This lesson covers the standards that set inspection, testing and maintenance requirements, the reliability reasoning behind test intervals, integrated testing, impairment management, hot work and construction fire safety, and the records that show a program is working.

Why standby systems fail

Sprinklers have an excellent record: studies of fires in the US by the National Fire Protection Association have found that sprinklers operate in about nine of ten fires large enough to activate them, and are effective in controlling the fire in the great majority of those cases. When sprinklers fail to operate, the most common reason is that the system had been shut off, followed by manual intervention that defeated it, lack of maintenance, and systems that were inappropriate for the hazard. In other words, most failures are failures of management rather than of hardware.

Similar patterns apply to other systems. Fire alarm systems fail because detectors are disabled during work and never restored, batteries are not replaced, or programming changes are not tested. Clean agent systems fail because cylinders lose pressure, release panels are left in manual or disabled mode, or new penetrations let the agent leak away. A hidden failure is one that is not revealed by normal operation and can only be found by inspection or testing, and the purpose of an inspection, testing and maintenance (ITM) program is to find hidden failures before a fire does.

Codes and standards for ITM

In Canada, the National Fire Code and provincial fire codes require owners to inspect, test and maintain fire protection systems, and reference standards for each system: NFPA 25 for water-based systems (sprinklers, standpipes, fire pumps, private mains and tanks), CAN/ULC-S536 for fire alarm systems, NFPA 2001 and the manufacturer's instructions for clean agent systems, NFPA 10 for portable extinguishers, and CAN/ULC-S1001 for integrated testing of interconnected systems. Emergency lighting is inspected and tested under the fire code. Work must be done by qualified people, and some provinces require fire alarm technicians to hold specific certifications.

The standards distinguish three kinds of activity. An inspection is a visual check that a component is in place, undamaged and in its normal condition (a valve is open, a gauge shows normal pressure). A test is an operation of the component to verify that it works (opening a test valve to check that the waterflow alarm sounds). Maintenance is work to keep the component in working order or restore it (replacing a battery, lubricating a valve).

Reliability and test intervals

Why test a valve annually rather than monthly or every five years? Reliability theory gives a useful answer. Suppose a component develops hidden failures at a constant rate \(\lambda\) per year, and a test every \(T\) years finds and repairs any failure. Immediately after a test, the component is known to work; the probability that it has failed grows during the interval and returns to zero at the next test. Averaged over the interval, the probability of failure on demand (PFD) is approximately:

valid when \(\lambda T\) is small. Halving the test interval halves the average unavailability. The formula also shows why tests that do not exercise every failure mode leave residual risk: failures that a test cannot reveal accumulate until a more thorough test, or forever if there is none.

import math

# Average probability of failure on demand (PFD) of a periodically tested standby component
def pfd_exact(lam, T):
    # average unavailability over a test interval with exponential failures
    x = lam * T
    return 1 - (1 - math.exp(-x)) / x

lam = 0.05            # undetected (hidden) failures per year, e.g. a pre-action valve assembly
print("test interval   simple lam*T/2   exact")
for months in (1, 3, 6, 12, 24, 60):
    T = months / 12
    print(f"{months:4d} months    {lam * T / 2:12.4f}   {pfd_exact(lam, T):7.4f}")

# Partial test coverage: an annual trip test reveals 90% of failure modes;
# the rest are found only by a full-flow test every 3 years.
c = 0.90
pfd = c * lam * 1 / 2 + (1 - c) * lam * 3 / 2
print(f"annual test with 90% coverage, full test every 3 years: PFD {pfd:.4f}")
pfd_never = c * lam * 1 / 2 + (1 - c) * lam * 20 / 2
print(f"if the remaining 10% is never tested (20-year life): PFD {pfd_never:.4f}")

test interval   simple lam*T/2   exact
   1 months          0.0021    0.0021
   3 months          0.0063    0.0062
   6 months          0.0125    0.0124
  12 months          0.0250    0.0246
  24 months          0.0500    0.0484
  60 months          0.1250    0.1152
annual test with 90% coverage, full test every 3 years: PFD 0.0300
if the remaining 10% is never tested (20-year life): PFD 0.0725

The output shows that with a hidden failure rate of 0.05 per year, annual testing gives an average PFD of about 0.025, meaning the component would fail to work in about one demand in forty. Testing monthly would reduce this to about one in 500, but every test has costs: labour, the risk of leaving the system impaired after the test, and wear on the equipment. The intervals in the standards reflect long experience of this balance. For critical components in a data centre, continuous supervision (valve tamper switches, pressure monitoring, battery monitoring) effectively converts hidden failures into revealed ones and is generally more valuable than more frequent manual testing.

Integrated testing

Each fire protection and life safety system may pass its own tests and still fail when they must work together. A detector may correctly put the fire alarm panel into alarm, but the relay that should shut down an air handler may have been wired to the wrong unit; the clean agent release panel may work, but the damper it closes may be stuck open; the access control system may be programmed to unlock egress doors on alarm, but a software update may have removed the rule. Integrated testing operates the systems together, following the sequence of operations matrix described in Lesson 4, and verifies every output from every input.

CAN/ULC-S1001 sets out the process for integrated systems testing, which the NBC and NFC require for new buildings and after modifications. The tests are coordinated by an integrated testing coordinator and documented in a report.

In data centres, integrated fire and life safety tests are usually part of the wider commissioning program that also tests power and cooling (DCE-450). Because many interfaces affect live IT systems (shutting down cooling or signalling the EPO), periodic integrated tests in operating facilities need careful planning, often testing outputs up to the point of operation with the final action blocked, then testing the final devices separately.

Impairment management

An impairment is any condition in which a fire protection system, or part of it, is out of service. Planned impairments occur during maintenance and modifications: draining a sprinkler zone to add a branch line, disabling detectors while work produces dust, or isolating a clean agent release during cylinder replacement. Emergency impairments occur unexpectedly: a broken pipe, a failed fire pump, a detection loop fault. Because the leading cause of sprinkler failure is a system that was shut off, managing impairments is among the most important parts of a fire safety program.

NFPA 25 and the fire codes require a formal impairment procedure, which typically includes:

• An impairment coordinator who authorizes and tracks all impairments.
• A tag placed at the fire department connection, the control valve and the fire alarm panel to show which system is impaired.
• Notification of the fire department, the monitoring station, the insurer and the people in the affected area before the impairment begins.
• Compensating measures: a fire watch (a trained person patrolling the unprotected area), suspension of hot work and other hazardous activities, temporary detection or extinguishers, or temporary water supplies.
• Minimizing the duration: preparing all materials before shutting a system down, and working continuously until it is restored.
• Restoration: returning the system to service, testing it (for example, a main drain test after a valve has been closed), removing tags and notifying everyone who was told of the impairment.

# Probability that a pre-action sprinkler system controls a fire, and the effect of impairments
# Each step must succeed (a series chain); probabilities are illustrative.
chain = [
    ("detection system signals the valve", 0.99),
    ("pre-action valve opens", 0.98),
    ("water supply available (control valve open)", 0.995),
    ("sprinklers operate and control the fire", 0.96),
]
p = 1.0
for name, ps in chain:
    p *= ps
    print(f"{name:<46} {ps:.3f}  cumulative {p:.4f}")
print(f"probability of control on demand: {p:.3f}; failure {1 - p:.3f}")

# Impairments: hours per year with the water supply shut for work, with and without good practice
for label, hours in (("well-managed impairments", 40), ("poorly managed impairments", 600)):
    unavail = hours / 8760
    p_imp = p * (1 - unavail)
    print(f"{label}: {hours} h per year shut -> control probability {p_imp:.4f} (failure {1 - p_imp:.4f})")

# Fire watch and other compensating measures can catch fires during impairments
watch_effect = 0.7          # fraction of fires found and controlled by a fire watch during impairment
hours = 600
p_with_watch = p * (1 - hours / 8760) + (hours / 8760) * watch_effect
print(f"poorly managed but with fire watch: control probability {p_with_watch:.4f}")

detection system signals the valve             0.990  cumulative 0.9900
pre-action valve opens                         0.980  cumulative 0.9702
water supply available (control valve open)    0.995  cumulative 0.9653
sprinklers operate and control the fire        0.960  cumulative 0.9267
probability of control on demand: 0.927; failure 0.073
well-managed impairments: 40 h per year shut -> control probability 0.9225 (failure 0.0775)
poorly managed impairments: 600 h per year shut -> control probability 0.8633 (failure 0.1367)
poorly managed but with fire watch: control probability 0.9112

The result shows how impairments erode protection. A system that is shut off for 600 hours a year, about 7 percent of the time, nearly doubles its probability of failing to control a fire, adding almost as much risk as all its hardware failures combined. Disciplined impairment management keeps the hours low, and a fire watch recovers much of the protection during the hours that remain. In a data centre undergoing continuous expansion and refits, the impairment program is a daily activity, not an occasional one.

Hot work and construction

Hot work (welding, cutting, brazing, soldering and grinding) is one of the most common causes of fires in buildings under construction or renovation. Data centres see a great deal of hot work: brazing refrigerant and chilled-water pipes, welding structural steel and pipe supports, and cutting during demolition. Sparks and hot slag can travel more than 10 m, fall through openings and smoulder unnoticed in concealed spaces before igniting after the workers have left.

In Canada, the NFC sets requirements for hot work, and CSA W117.2 covers safety in welding, cutting and allied processes. A hot work permit system, often based on NFPA 51B as well, is standard practice in data centres.

A hot work permit system requires that someone other than the worker inspect the area before work starts and authorize it in writing. Typical requirements are that combustibles within about 11 m (35 ft) be removed or protected, openings in floors and walls covered, sprinklers and detection remain in service where possible, an extinguisher be at hand, and a fire watch remain during the work and for at least one hour afterwards, as required since the 2019 edition of NFPA 51B, with further monitoring for up to three hours where the permit authorizer requires it. Where hot work is planned to avoid it in the first place (for example, using mechanical press-fit pipe joints instead of brazing), the risk disappears.

Hot work and other construction activities also generate smoke, dust and fumes that trigger sensitive aspirating detectors. The temptation to disable detection for convenience must be resisted: the approved approach is a planned impairment of the smallest possible zone, with compensating measures, and restoration as soon as the work stops each day. Some sites use temporary local detectors or adjust aspirating thresholds in the affected sector only, under the impairment procedure.

Buildings under construction are at their most vulnerable, with combustible materials, temporary heating, hot work and incomplete fire protection. The NBC and NFC include requirements for fire safety at construction and demolition sites, including a fire safety plan for the site, access for firefighters, water supplies available as soon as combustible materials arrive, and standpipes that are extended as the building rises.

Housekeeping and change control

Many fire risks grow slowly through small changes: a stack of packaging left in a corridor, a new cable bundle through a fire separation without fire-stopping, a ceiling tile removed and not replaced above a clean agent room, a rack added that blocks a sprinkler or a detector sampling point. Regular walk-through inspections with a checklist, and a change control process in which every change is reviewed for its effect on fire protection, keep these risks in check. Abandoned cables, which add fuel and obstruct airflow and inspection, should be removed; electrical codes require accessible abandoned cables to be removed in many spaces.

Records and performance

Every inspection, test, maintenance task, deficiency and impairment should be recorded, with the date, the person, the result and any corrective action. Records demonstrate compliance to the authority having jurisdiction, insurers and customers, and they reveal trends: a main drain test whose residual pressure has fallen over several years may indicate a partly closed valve or a deteriorating supply; a detector that often needs cleaning may be poorly located. Useful program measures include the percentage of scheduled ITM tasks completed on time, the number and age of open deficiencies, the number and duration of impairments, and the rate of unwanted alarms. Reviewing these at management level keeps fire protection visible in an organization whose attention is naturally on power and cooling.