PWR-340 · Lesson 10 of 12 · 23 min

Incident investigation in plants

▶ Video coming soon A video version of this lesson is on the way.

On November 14, 2026, the laboratory's report on the tube samples from boiler 3 arrived (PWR-220). It confirmed what Grazyna had suspected since the leak on November 2: the floor tube had failed by oxygen pitting that started during the summer wet layup, when the boiler stood full of water for weeks. The report described the pits, their depth and their oxide, and it listed the deepest remaining pit on a neighboring tube at 0.063 in. It did not say why oxygen had got in, or why nobody had noticed.

"The lab tells you what happened to the steel," Grazyna told Okonkwo. "An investigation tells you what happened to us. Those are different questions, and the second one is the one that stops it happening again." She had begun drafting the investigation the day the report arrived, and she asked Okonkwo to help her finish it before the November operations meeting.

This lesson explains why incidents and near misses are investigated, what must be reported and to whom, how an investigation is organized, how evidence is gathered and a timeline built, and how causes are analyzed with methods such as five whys, cause and effect diagrams, barrier analysis and causal factor charts. It discusses human factors and why "operator error" is never a root cause, how recommendations are written and tracked, and how the lessons are shared. It works through the boiler 3 pitting investigation and a calculation of an injury frequency rate.

Why investigate

Every incident and near miss is information that the plant has paid for and will waste unless it learns from it. The purpose of an investigation is prevention: to understand the causes well enough to fix them, not to assign blame. Investigations look at near misses as well as incidents, because near misses are far more frequent, cost nothing to learn from, and usually share the same underlying causes. A plant that investigates only its injuries is learning from the smallest and most expensive part of its experience.

Investigation is also a legal duty. Process safety rules require incidents that resulted in, or could reasonably have resulted in, a catastrophic release to be investigated promptly; OSHA's standard requires the investigation to start within 48 hours. Pressure equipment and workplace safety laws require some events to be reported to the regulator quickly, and they set out what must be preserved.

What must be reported

In Ohio, a stationary engineer deals with several reporting duties:

  • The Ohio Division of Industrial Compliance must be notified of boiler accidents under the state's boiler rules (OAC 1301:3-5-04, on unsafe conditions and accidents), and the scene must not be disturbed, except to save life or prevent further harm, until the inspector allows. The boiler 3 tube leak was handled through the insurance company's special inspector, and Ottoline Brackwater agreed the repair plan on November 3.
  • OSHA must be told of a work-related fatality within 8 hours, and of an in-patient hospitalization, amputation or loss of an eye within 24 hours, under 29 CFR 1904.39. Recordable injuries and illnesses are entered on the plant's OSHA logs.
  • Environmental authorities must be told of spills and releases above set limits, through the provincial or state spill line and, in some cases, the national response center, as PWR-240 described.
  • The insurer and the joint health and safety committee or plant safety committee must be informed under the policy and the plant's own rules.

The plant's emergency procedure lists these contacts and the deadlines, because in the hours after a serious incident nobody should have to work them out.

Organizing an investigation

The size of an investigation should match the actual or potential severity of the event, not just its actual outcome. A near miss that could have killed someone deserves a serious investigation even though nobody was hurt. A typical investigation has:

  • A lead investigator trained in the method, independent of the area where the event happened if possible.
  • A team with people who know the equipment and the work, including an operator or worker from the area, and specialists such as an engineer, a metallurgist or a controls technician where needed. Worker participation is required by process safety rules and is good practice everywhere.
  • Terms of reference stating what is to be investigated, the deadline and who receives the report.

The team's first jobs are to make the scene safe and to preserve evidence before it is lost: photographs, physical parts (the failed tube, the gasket, the valve), data from the control system and historian, logbooks, permits and procedures as they were on the day, and the names of witnesses. Memories fade and change quickly, so witnesses are interviewed as soon as practical, one at a time, with open questions and without judgment.

Building the timeline

The backbone of most investigations is a timeline or sequence of events: what happened, when, in what order, with each item tied to its evidence. Conditions that existed before the event go alongside the events, because the conditions are often where the causes are.

For boiler 3, the timeline reached back to June:

Part of the boiler 3 pitting timeline.
DateEvent or conditionEvidence
Early MayBoiler 3 placed in summer wet layup as the summer standby; water treated at shutdown; procedure calls for the scavenger residual to be maintainedShift log, layup procedure
JuneLast sulfite residual test of boiler 3 layup water recordedWater treatment log
July to AugustNo further layup water tests recorded; the layup tests were not on the shift task list, so they depended on someone remembering themTask list, water treatment log
August 17 to 18Drum manway opened and closed during layup, one of two openings that summer; air admitted, and the water not retreated afterwardWork order, shift log
September 15 to 19Drum entered and boiler inspected, then returned to service (PWR-110, PWR-120); the layup record was not part of the return-to-service check, and the floor tubes, the last to drain, were not examined closely for pittingInspection report, shift log
November 2, 01:30Desmond Kirwan finds about 373 lb/h of water unaccounted for; boiler 3 taken off lineWater balance, shift log
November 3 to 5Pinhole found in a furnace floor tube near the rear wall; tube replaced and hydrostatic test witnessedRepair records
November 14Laboratory report: oxygen pitting from layup; deepest remaining pit on a neighboring tube 0.063 inLab report

Analyzing causes

With the facts laid out, the team works out why each event happened. Investigators distinguish three levels of cause:

  • The direct or immediate cause: the event or condition that directly produced the harm. For boiler 3, oxygen pitting of a floor tube.
  • Contributing or causal factors: the events and conditions without which the incident would not have happened or would have been less severe. For boiler 3: dissolved oxygen in the layup water; a scavenger residual that was allowed to fall untested; air admitted when the drum was opened, without retreatment; a return to service that did not look at the layup record.
  • Root causes: the underlying failures in the management system that allowed the causal factors to exist, and which, if corrected, would prevent this and similar incidents. For boiler 3: layup tests that lived in an old procedure and had never been put on the shift task list, where every other routine test is scheduled and logged; no requirement in the procedure to retreat the water after the boiler was opened; and no check of layup chemistry in the return-to-service procedure.

Several methods help the team get from direct causes to root causes:

  • Five whys: asking "why?" repeatedly, typically about five times, until the answer is a management system issue. It is simple and quick but can follow only one chain at a time and depends on the team's knowledge.
  • Cause and effect or fishbone diagrams: grouping possible causes under headings such as people, procedures, equipment, materials, environment and management, so that no category is overlooked.
  • Barrier analysis: listing the barriers that should have prevented the event and asking, for each, whether it existed, whether it worked and why not.
  • Causal factor charts and structured methods such as fault trees, TapRooT and the Apollo method, used for complex events.

Worked problem

Five whys for boiler 3

Apply five whys to the pitting of boiler 3's floor tubes.

Show solution
  1. Why did the tube fail? Oxygen pitting penetrated the wall.
  2. Why was there oxygen pitting? The layup water contained dissolved oxygen for weeks without enough oxygen scavenger.
  3. Why was there not enough scavenger? The sulfite residual was not tested after June, and air was let in when the drum was opened in August without the water being retreated.
  4. Why was it not tested or retreated? The layup tests were not on the shift task list, and the procedure did not require retreatment after the drum was opened.
  5. Why did the procedure and task list have those gaps? When the plant moved its routine tests onto the shift task list, the layup procedure, used only once a year, was overlooked; and the return-to-service procedure had no check of layup chemistry.

Answer: Five whys reach a management system cause: a procedure left behind when the plant changed how it schedules routine work, and a missing check. Moving the plant's tests onto a new task list was itself a change, and an MOC review of it (lesson 9) might have caught the gap.

A row of slices representing the barriers against layup corrosion: layup procedure, weekly residual test, retreatment after opening, shift task list, and return-to-service check. Each slice has holes labeled with what failed: procedure not reviewed, test not done after June, no retreatment step, tests not on the list, no layup check. An arrow passes through aligned holes to the pitted tube on the right.
The barriers that failed in the boiler 3 layup, drawn as layers with holes.

Human factors

It is tempting to stop an investigation when a person is found who did something wrong: the operator who missed the test, the mechanic who topped up the drum. That is almost never a root cause. People's actions are shaped by the procedures, training, workload, equipment design, supervision and culture around them. The useful question is not "who failed?" but "why did it make sense to them at the time, and what would make the right action easier and the wrong one harder?"

Investigators distinguish slips and lapses, unintended actions or omissions by someone who knows what to do, which are reduced by better design, checklists and reminders; mistakes, where the person's plan was wrong because of incomplete knowledge or a misleading situation, reduced by training and better information; and violations, deliberate departures from rules, which are themselves usually symptoms of rules that are unworkable or of norms that tolerate them. A just culture treats honest errors as system problems while still holding people accountable for reckless behavior. Without it, near misses stop being reported, and the plant goes blind.

Recommendations and follow-through

Recommendations should address the root causes, be specific, assign an owner and a due date, and be checked for effectiveness after they are done. Stronger recommendations follow the hierarchy of controls: eliminating a hazard is better than engineering it out, which is better than administrative controls such as procedures and training, which are better than personal protective equipment. A recommendation that says "remind operators to be careful" is weak; one that adds a required field to the return-to-service checklist, with the residual value recorded before the boiler can be fired, is stronger.

The boiler 3 investigation's draft recommendations, going to the November operations meeting, are: revise the wet layup procedure (full flooding, higher scavenger residual and pH, weekly tests on the shift task list, retreatment after any opening, and a nitrogen cap), with a dry layup option for long outages (PWR-220); add layup chemistry to the return-to-service checklist; review every procedure used only seasonally to make sure its routine tests are on the task list; and replace the deepest-pitted floor tubes at the planned 2027 outage, with ultrasonic thickness mapping of the rest. Each has an owner and a date, and Grazyna will report on them at each monthly meeting until they are closed.

Lessons are then shared: with every shift, in a short summary at the start of each briefing; with the joint health and safety committee or plant safety committee; and, without identifying individuals, with other plants. Organizations such as the CSB publish their investigations and safety videos for this reason, and many of the best lessons in this course come from them.

Measuring incidents

Injury statistics are lagging indicators (lesson 8) but still useful when used properly. The most common is the total recordable incident rate (TRIR), normalized to 200,000 hours, the hours worked by about 100 full-time workers in a year:

\[\text{TRIR} = \frac{\text{number of recordable incidents} \times 200{,}000}{\text{total hours worked}}\]

Worked problem

Cantwell's rate

Cantwell's staff worked about 96,000 hours in the last twelve months and had two recordable injuries, a sprained wrist from a slip on the condensate tank ladder and a back strain while handling a chemical drum. Okonkwo's cut hand in October (PWR-240) was treated with first aid only and is not recordable. Find the TRIR, and the rate for the one reported near miss involving pressure equipment in the same period.

Show solution
  1. \[\text{TRIR} = \frac{2 \times 200{,}000}{96{,}000} = 4.17\]
  2. \[\text{near miss rate} = \frac{1 \times 200{,}000}{96{,}000} = 2.08\]

Answer: With so few hours, one injury moves the rate a long way, so a small plant's rate is not meaningful from year to year. More telling is that the pressure equipment near miss rate is lower than the injury rate: near misses should outnumber injuries many times over, so a low count probably means near misses are not being reported, not that they are not happening. Grazyna put that on the agenda too.

Common mistakes

  • MistakeStopping the investigation at "operator error."

    InsteadAsk why the action made sense at the time and fix the system that shaped it.

  • MistakeInvestigating only incidents that caused injury.

    InsteadInvestigate near misses and events by their potential severity.

  • MistakeMoving failed parts or clearing the scene before evidence is recorded.

    InsteadMake safe, then preserve evidence and data first, subject to the regulator's rules.

  • MistakeWriting recommendations such as "be more careful."

    InsteadWrite specific actions with owners and dates, high on the hierarchy of controls.

  • MistakeClosing a recommendation when it is issued rather than when it is effective.

    InsteadTrack it to completion and check that it worked.

  • MistakeMissing a regulatory reporting deadline while the investigation starts.

    InsteadReport first, as the procedure lists, then investigate.

Summary and key terms

  • Investigations aim at prevention, not blame, and include near misses judged by their potential severity.
  • Regulators, insurers and safety committees must be notified of certain events within set times; preserve the scene and evidence.
  • A timeline of events and conditions, tied to evidence, is the backbone of an investigation.
  • Direct causes, causal factors and root causes are distinguished; five whys, fishbones, barrier analysis and structured methods help find root causes.
  • Human error is a starting point, not a root cause; slips, mistakes and violations each have system causes, and a just culture keeps reporting alive.
  • Recommendations address root causes, follow the hierarchy of controls, have owners and dates, and are tracked to effectiveness and shared.
Near miss
An event that could have caused harm but did not.
Direct cause
The event or condition that directly produced the harm.
Causal factor
An event or condition without which the incident would not have happened or would have been less severe.
Root cause
An underlying management system failure that, if corrected, prevents recurrence of this and similar incidents.
Five whys
A method of finding root causes by repeatedly asking why.
Barrier analysis
A method that examines which barriers should have prevented an event and why they failed.
Just culture
An approach that treats honest errors as system problems while holding people accountable for reckless behavior.
Hierarchy of controls
The ranking of controls from elimination and engineering down to administrative controls and protective equipment.
Total recordable incident rate
Recordable incidents per 200,000 hours worked.

Practice problems

Work each problem on paper first, then reveal the answer to check.

  1. A plant worked 150,000 hours with 3 recordable incidents. TRIR?

    Show answer

    4.0.

  2. Is "the operator forgot to check the level" a root cause?

    Show answer

    No; ask why the check was missed and what in the system allowed it.

  3. Which is the stronger recommendation: retraining staff, or adding an interlock that prevents the error?

    Show answer

    The interlock, higher on the hierarchy of controls.

  4. Name the three levels of cause an investigation distinguishes.

    Show answer

    Direct cause, causal factors and root causes.

Lesson quiz · 5 questions

Score 60% or more to complete this lesson.

Take the quiz