Functional performance testing and integrated systems testing

At 2:00 on a Wednesday afternoon in March, Kwame Asante stood in the East Wing mechanical room with a printed test procedure, a radio and a laptop showing the building automation system. 'Step 14,' he said. 'Simulate failure of supply fan 2.' Helen's controls technician opened the disconnect on one of the four fans in the new air handling unit's fan array. Kwame watched the screen. Within ten seconds the remaining three fans should have ramped up to hold duct pressure, an alarm should have appeared at the operators' workstation, and the dialysis unit room pressures should have stayed positive. The fans ramped up. The duct pressure dipped and recovered in fifteen seconds. The alarm appeared, but it said 'AHU-3 general fault' instead of 'AHU-3 supply fan 2 failure', which would have sent a night operator to the wrong place. Kwame wrote 'fail: alarm text' on the sheet, logged it as issue 211, and moved on to step 15. That is functional performance testing: proving, one condition at a time, that systems do what the owner needs, including when something goes wrong.

What functional testing is

Functional performance testing, often shortened to functional testing or FPT, checks that equipment and systems operate according to the approved sequences of operation and the owner's project requirements, in all the modes they will see in real life. Construction checklists confirm that things are installed correctly; functional tests confirm that they behave correctly. A properly installed air handling unit with a wrong control sequence will pass every checklist and fail its functional test.

Functional tests cover normal operation, but the most valuable steps test the conditions that rarely happen: equipment failures, loss of power, fire alarm activation, extreme weather, loss of communication with the automation system, sensor failures. Those conditions are exactly the ones in which a hospital most needs its systems to work, and exactly the ones nobody sees in daily operation until a real emergency.

Writing test procedures

Kwame's team wrote a test procedure for every system type. Each procedure is a script: it lists prerequisites, then numbered steps, each with an action, the expected response drawn from the sequence of operation, a space for the actual response, and a pass or fail mark. Procedures are sent to the contractor and designers before testing so that everyone agrees what will be tested and what counts as a pass. Disagreements about the expected response are much easier to resolve at a desk than in a mechanical room with ten people watching.

Methods: watching, forcing and trending

Some steps can be tested by watching the system respond to a real change, such as switching the unit to occupied mode. Others need the tester to force a condition: overriding a sensor value in the automation system, opening a disconnect, closing a valve or heating a sensor with a heat gun. Forcing has to be done carefully, because a forced value left in place after the test can cause problems for months. On Ashgrove every override was recorded on the test sheet and released before the test was signed.

Trend logging is the third method. The automation system records values such as temperatures, pressures, valve positions and fan speeds every few minutes over days or weeks. Trend data shows how systems behave across real cycles of occupancy and weather, catching problems that a two-hour test cannot, such as valves hunting back and forth, simultaneous heating and cooling, or rooms that overheat every afternoon. Kwame required two weeks of trend data from the dialysis unit before it opened and reviewed it with the controls subcontractor. It showed that two reheat valves were opening while the cooling coil was at full capacity: the units were heating and cooling the same air at the same time, wasting energy, because of a reversed control signal.

Sampling: test everything or test a sample?

Critical equipment, such as air handling units, generators, fire pumps and pressure-critical rooms, is usually tested in full. Repetitive equipment, such as dozens of identical reheat terminals, may be tested by sampling: a random sample is tested, and if the failure rate in the sample is above an agreed threshold, a further sample is tested, and if that also fails, the contractor must test and correct every unit at its own cost. Sampling saves time while still giving the owner confidence, and the threat of full testing encourages the contractor to get things right the first time.

Tomas wanted to know how likely Harborline was to end up testing every terminal if its installers had a given error rate, so he wrote a quick simulation. It showed that at a true failure rate of 5 %, full testing was unlikely; at 15 %, it happened in about half of all trials. He showed the result to the controls subcontractor's foreman as a reason to check every terminal before the commissioning authority arrived.

# Probability that a sampling rule forces full testing
import random
random.seed(7)
N, frac, limit = 64, 0.20, 0.10
n = -(-int(N * frac * 10) // 10)        # 20 % of 64, rounded up = 13

def full_test(rate):
    units = [random.random() < rate for _ in range(N)]
    random.shuffle(units)
    s1, s2 = units[:n], units[n:2 * n]
    return sum(s1) / n > limit and sum(s2) / n > limit

for rate in (0.02, 0.05, 0.10, 0.15, 0.25):
    trials = 20000
    p = sum(full_test(rate) for _ in range(trials)) / trials
    print(f'True failure rate {rate:4.0%}: full testing in {p:5.1%} of trials')

True failure rate   2%: full testing in  0.1% of trials
True failure rate   5%: full testing in  1.9% of trials
True failure rate  10%: full testing in 14.1% of trials
True failure rate  15%: full testing in 35.9% of trials
True failure rate  25%: full testing in 76.6% of trials

Emergency power and other critical systems

In a hospital, the emergency power system is one of the most important systems to test, because it keeps life safety and critical care equipment running when normal power fails. In the United States, NFPA 99, the Health Care Facilities Code, and NFPA 110, the Standard for Emergency and Standby Power Systems, set requirements for essential electrical systems and their testing, along with the electrical code's provisions. On Ashgrove, the new air handling unit, the dialysis unit's water treatment system and part of the lighting were connected to the hospital's existing generators through new transfer switches. Functional testing included a full transfer test: Helen's electricians opened the normal power supply, and the commissioning team recorded the time for each transfer switch to transfer, each fan to restart and each critical load to recover.

The test had to be scheduled with the hospital's clinical leaders, because a transfer affects every load on the switches involved. It was done at 6:00 on a Sunday, with the dialysis unit not yet open and the Main Building loads on separate switches. The test found one transfer switch with a time-delay setting longer than the design, which would have left the dialysis water treatment system without power for nearly a minute. The setting was corrected and the test repeated the following Sunday.

Integrated systems testing

Integrated systems testing checks that separate systems work together as designed, especially for life safety. When the fire alarm detects smoke, the air handling unit should stop or go into smoke control mode, smoke dampers should close, door hold-open devices should release, elevators should recall and the hospital's switchboard should receive the right signal. Each system may pass its own test and still fail in combination, because one contractor assumed the other would provide a relay or a signal. Lesson 9 covers fire alarm and life safety testing in detail; the important point here is that integrated testing needs every trade involved present at once, with a single procedure and a single person in charge of the test.

When tests fail

Failures are expected; that is why tests are done. What matters is how they are handled. Every failed step on Ashgrove became an entry in the issues log with a description, the responsible party and a target date. The responsible party corrected the problem and told the commissioning authority, who then retested the step, or the whole test if the correction could have affected other steps. Most contracts, including Ashgrove's, allow the owner to charge the contractor for the commissioning authority's time on repeated retests caused by the contractor's failure to be ready, which concentrates minds.

The generic alarm text found in step 14 took the controls subcontractor two hours to fix: the alarm points had been named from a template that had never been edited. Kwame retested steps 14 to 16 the following week, and they passed. A check of the other alarm points found 37 more with generic text, all corrected before handover. One failed step had exposed a pattern.