Safety cases and licensing documents

Dr. Rohan Desai leads safety analysis at Meridian Nuclear Engineering. When Hana joins his team for a week, he shows her a wall of shelves holding the safety report for a single station, more than forty volumes. "People think a safety case is this," he says, tapping a binder. "It is not. This is the evidence. The safety case is the argument that connects the evidence to the claim that the plant is safe enough. If you cannot state the argument on one page, you do not understand your own safety case."

What a safety case is

A safety case is a structured argument, supported by evidence, that a facility or activity is acceptably safe in a given context. The idea comes from several high-hazard industries and is used in nuclear regulation around the world. It has three parts. A claim is a statement about the facility that must be true for it to be safe, such as "the core remains cooled after any design basis accident." An argument explains why the claim is true, for example by showing that the emergency core cooling system has enough capacity and redundancy. Evidence supports the argument: analysis results, test data, operating experience, inspection records and qualification reports. This framework is often called claims, arguments and evidence.

A good safety case is explicit about its assumptions, honest about its uncertainties and complete. It covers the whole life of the facility, not just normal operation. It shows that hazards have been identified systematically, that defence in depth is in place, that safety margins exist, and that the residual risk is acceptable. It is also a living document: it must be updated when the facility, the knowledge base or the requirements change. A safety case written once and never revised becomes a description of a plant that no longer exists.

The deterministic and probabilistic parts

The technical heart of a reactor safety case has two complementary parts. The deterministic safety analysis takes a defined set of events, such as a pipe break, a loss of power or a failure of a control system, and shows by conservative or best-estimate-plus-uncertainty calculation that the plant meets acceptance criteria: fuel temperatures stay below limits, the pressure boundary holds and doses to the public stay below limits. The events are grouped by frequency into anticipated operational occurrences, design basis accidents and design extension conditions, with stricter criteria for more frequent events. The probabilistic safety assessment looks at the full range of possible event sequences, estimates their frequencies and consequences, and reports summary measures such as core damage frequency and large release frequency. It shows where the risk comes from and whether the plant meets safety goals.

In Canada, REGDOC-2.4.1 sets out requirements for deterministic safety analysis and REGDOC-2.4.2 for probabilistic safety assessment. The CNSC's safety goals for new reactors include a core damage frequency below one in a hundred thousand per reactor-year, a small release frequency below one in a hundred thousand and a large release frequency below one in a million. Existing plants are expected to meet comparable limits and targets set out in their licensing basis.

The licensing documents

For a power reactor, the main licensing document is the safety report. It typically has three parts: a description of the plant and site; the safety analysis, including the deterministic analyses of design basis accidents and design extension conditions; and the probabilistic safety assessment. The licensee also submits programme documents for each safety and control area, such as the radiation protection programme, the fire protection programme and the emergency plan. CNSC staff review these and report their conclusions in a Commission member document, which goes to the Commission with a recommendation before the hearing.

Supporting the main report are hundreds of technical reports: analyses of particular events, qualification reports for equipment, design manuals, code validation reports and so on. The safety report summarizes and references them. A reviewer cannot read everything, so the structure of the report, and the clarity of the argument, determine how efficiently the review goes. Rohan tells Hana that the best safety reports are written for a reader who is intelligent, sceptical and short of time.

Finding the gaps

A reviewer reading a safety case asks, for every claim, whether the argument is sound and the evidence sufficient. Rohan's team uses a simple tool when preparing an application: they list every claim, sub-claim, argument and evidence item in a structured form and check that each claim is supported. A claim with no evidence is a gap; an evidence item that supports no claim may be unnecessary or may reveal a missing claim. Counting does not replace judgement, but it makes gaps visible early.

# Walk a claims-arguments-evidence tree, count items and find unsupported claims.
tree = ("C", "plant is acceptably safe", [
    ("C", "reactivity is controlled", [
        ("A", "two independent shutdown systems", [
            ("E", "shutdown system test reports", []), ("E", "reactivity transient analysis", [])])]),
    ("C", "fuel remains cooled", [
        ("A", "emergency core cooling has margin", [
            ("E", "loss of coolant analysis", []), ("E", "pump qualification report", [])])]),
    ("C", "radioactivity is contained", [
        ("C", "containment pressure stays below design", [
            ("A", "containment design and analysis", [("E", "containment analysis", []), ("E", "leak rate test", [])])]),
        ("C", "hydrogen is managed", [
            ("A", "recombiners limit concentration", [])])]),
])
count = {"C": 0, "A": 0, "E": 0}; gaps = []
def has_evidence(node):
    kind, text, kids = node
    return kind == "E" or any(has_evidence(k) for k in kids)
def walk(node, depth=0):
    kind, text, kids = node
    count[kind] += 1
    if kind == "C" and not has_evidence(node): gaps.append(text)
    for k in kids: walk(k, depth + 1)
walk(tree)
print("claims", count["C"], "arguments", count["A"], "evidence", count["E"])
print("claims without evidence:", gaps if gaps else "none")

claims 6 arguments 4 evidence 6
claims without evidence: ['hydrogen is managed']

The output flags the claim about hydrogen management, which has an argument but no evidence yet. Before the application is submitted, the team will need a hydrogen distribution analysis and qualification data for the recombiners. Finding this gap now costs a few months of work; finding it during the regulator's review would cost a request for additional information, a delay and some loss of the reviewer's confidence.

Review and requests for information

CNSC review follows a plan agreed with the applicant. Staff specialists in each area assess the submissions against the regulatory requirements and guidance, identify issues and send requests for information. The applicant responds, and the cycle repeats until the issues are resolved or deferred to a later stage with a clear plan. Some issues may become licence conditions or hold points. The staff's conclusions and any remaining issues are set out in the Commission member document. Members can and do ask hard questions about these at the hearing.

Living safety cases and periodic review

A safety case must keep pace with the plant. Licensees update the safety report on a regular cycle and whenever the design or operation changes significantly. Every ten years, under REGDOC-2.3.3, they carry out a periodic safety review that compares the plant with modern codes, standards and practices and with operating experience, identifies gaps and proposes improvements. The periodic safety review is the main input to licence renewal.

Defence in depth in the safety case

Every reactor safety case is organized, explicitly or not, around defence in depth. The IAEA describes five levels. The first is to prevent abnormal operation and failures through conservative design and high-quality construction and operation. The second is to control abnormal operation and detect failures, using control systems, limits and surveillance. The third is to control accidents within the design basis, using engineered safety features and procedures. The fourth is to control severe conditions, including preventing accidents from progressing and mitigating their consequences, chiefly by protecting the containment. The fifth is to mitigate the radiological consequences of a significant release through off-site emergency response.

A safety case shows that each level is in place and that the levels are as independent as practicable, so that a failure at one level does not defeat the next. A reviewer looks especially for common causes that could defeat several levels at once, such as a flood that disables both normal and emergency power, a fire that damages cables for redundant trains, or a software error copied into two systems. The Fukushima accident was a lesson in exactly this: the tsunami defeated levels two, three and four together because their equipment shared the same vulnerability to flooding.

The quality of the evidence

Evidence is only as good as the methods that produced it. Much of the evidence in a reactor safety case comes from computer codes that model thermal-hydraulics, reactor physics, fuel behaviour and containment response. A reviewer will ask whether each code has been validated against experiments relevant to the conditions being analysed, whether its uncertainties are known and accounted for, whether it was developed and maintained under a software quality assurance programme, and whether the analysts using it were trained and their work checked. A beautifully argued safety case resting on an unvalidated code is a weak safety case.

CNSC staff often carry out independent confirmatory analyses using their own codes, or commission them from technical support organizations, to check key results. The CNSC also funds research to maintain its expertise and to resolve generic safety issues. For CANDU reactors, a long list of such issues, known as the CANDU safety issues and ranked into categories by their significance, was worked through over many years, including questions about large loss of coolant accidents and positive void reactivity, with the results built into licensees' safety cases.

How safety cases fail

Investigations of accidents in other industries have identified ways in which safety cases can fail. The best known is the 2009 review by Charles Haddon-Cave into the loss of a military aircraft in Afghanistan, which found that the aircraft's safety case had become a paperwork exercise, produced to satisfy a requirement rather than to understand the hazards, and that it had missed a fire risk that had been present for years. The review warned against safety cases that are too long to read, that assume things are safe because they have not yet failed, and that are written by consultants detached from the people who operate the equipment.

Rohan takes these lessons seriously. He asks every analyst on his team to be able to explain, in plain words, what each part of the safety case claims and why it is true. He insists that operators and maintenance staff review the parts that concern their work, because they know where the plant differs from the drawings. And he encourages the team to look for evidence that the safety case is wrong, not only for evidence that it is right. "A reviewer who finds a gap is doing us a favour," he tells Hana. "A gap nobody finds is the one that hurts you."

Graded safety cases

Not every facility needs a forty-volume safety report. Regulators in both countries apply a graded approach, scaling the depth of the safety case to the hazard. A small research reactor with a low power and a small inventory may have a safety case of a few hundred pages, with simple bounding analyses showing that even the worst credible accident cannot cause significant off-site doses. A large power reactor needs the full set of analyses. Many small modular reactor designs aim to show, through passive safety features and small inventories, that the consequences of accidents are much smaller, which would justify a smaller emergency planning zone and a simpler licensing basis. The burden is on the designer to prove it, with evidence.

Rohan's final advice to Hana is about tone. A safety case should be neither defensive nor promotional. It should state what is known, what is assumed and what is uncertain, and it should make the reviewer's job easy by pointing to the evidence for each claim. "Write it as if the reader's job is to find the hole in it," he says, "because it is."