Design review

In week 19 the team presents Kestrel-10 to a design review board. Lise Fortin chairs it. With her are Marcel Thibodeau, a former reactor operator and shift supervisor who now reviews designs for operability, Elena Marchetti for licensing, Rohan Desai for physics and safety, Professor Mwangi, and Grant Whitford as the client. The board has had the design review package for a week. Lise opens the meeting by explaining what it is for: “We are not here to grade you. We are here to find the problems in this design while they are still cheap to fix. If we find nothing, either the design is perfect or we have not done our job, and I have never seen a perfect design.”

This lesson describes the purpose and types of design review, how a review package is prepared and a review conducted, how comments are recorded, ranked and dispositioned, and how a team tracks actions to closure. It follows the Kestrel-10 review from its preparation to the decision that allows the team to proceed to its final report, and it discusses the human side of being reviewed.

Why designs are reviewed

Every engineering organization that designs safety-significant systems uses independent design review, and nuclear quality assurance standards require it. The reason is simple: designers are the people least able to see the flaws in their own work. They know what they meant, so they read their drawings and calculations as they intended them rather than as they are. They share assumptions with their immediate colleagues, so a team can be collectively blind to a problem. And they are invested in the design, which makes them slower to accept that part of it is wrong. A reviewer who did not do the work, and who brings different experience, sees the design as others will see it: a regulator, a constructor, an operator, an inspector.

Reviews come in several forms. Checking verifies a calculation line by line, and is done on every calculation package by the independent reviewer named in the team's responsibility matrix. Peer review asks a qualified colleague to examine a design or analysis for its approach and assumptions as well as its arithmetic. Independent design review, which is what the board conducts, examines the whole design at a defined stage against its requirements and against the experience of people who have built and operated similar plants. Specialist reviews look at one aspect in depth: constructability, operability and maintainability, human factors, security, or fire protection. And gate reviews, like the concept selection gate, decide whether a project may proceed to its next stage.

Preparing the review package

A design review is only as good as its preparation. The team's review package contains the requirements document with the traceability matrix, the concept selection report, the calculation packages for neutronics, thermal hydraulics, safety analysis, shielding and cost, the licensing strategy, a list of open issues and assumptions, and a short design description that ties them together. It is distributed a week before the meeting, with a request that board members send written comments in advance, so that the meeting can be spent on the comments that need discussion rather than on reading.

Hana adds one item that Rohan recommends: a list of the questions the team itself is least sure about. It is an uncomfortable thing to write, because it points reviewers at the weakest parts of the design, but that is precisely its value. Her list includes the peaking factor used in the boiling margin, the warm surface layer design, the seismic behavior of the chimney, the staffing assumptions and the security design. Lise later tells the team that this list was the most useful page in the package.

Each board member reviews against criteria. Lise uses the requirements and the safety analysis acceptance criteria. Marcel uses an operability checklist: can the plant be started up, operated, shut down, refueled and maintained by the planned staff, with the planned instruments and procedures, without exposing anyone to unnecessary dose or hazard? Elena uses the compliance matrix. Rohan checks the physics and the consistency between areas. Grant asks whether the design meets the university's needs and whether the result is presented honestly.

Conducting the review

The meeting follows a fixed structure. The team presents the design in about forty minutes, organized by requirement rather than by discipline, so that the board can see how each requirement is met. The board then works through its written comments, one at a time. For each, the reviewer states the concern, the team responds with facts rather than defense, and the chair decides how the comment will be recorded: as a finding that requires action, as an observation for the team to consider, or as resolved in the meeting if the team's answer satisfies the reviewer. Findings are given an owner and a due date before the meeting moves on.

Several findings come from the interfaces between areas. Rohan notices that the boiling margin depends on a peaking factor of 1.9 that came from a judgment about gadolinia placement, not from a calculation; if the real peaking were 2.1, the margin would fall below the 10 K requirement. That becomes DR-01. Marcel asks how operators will know that the warm surface layer is working, since nitrogen-16 dose at the pool bridge depends on it; the team has described the layer only qualitatively, which becomes DR-07. Mira's chimney has not been checked for seismic loads, and a chimney that cracked or shifted in an earthquake could disturb natural circulation and the control rod guides: DR-03. Elena points out that the security and safeguards design is missing entirely: DR-08. Grant asks whether the cost of heat includes contributions to the decommissioning fund, and it does not: DR-05.

Other findings are smaller but useful. The pool liner leak detection system is described in the text but not shown on the layout drawing. The rod drop time used in the rod withdrawal analysis was assumed rather than justified. Two requirements, R-07 and R-12, have no verification method entered in the traceability matrix. And the decay heat calculation assumes three years at full power without saying that this is conservative compared with the expected operating history.

Ranking and tracking actions

Not all findings are equally important, and a team with limited time must work on the most important first. The board ranks each finding by the severity of its consequence if left unresolved and the likelihood that it reflects a real problem, each on a scale of 1 to 5. The product is the risk score: 12 or more is high, 6 to 11 medium and 5 or less low. The scoring is a judgment made in the meeting, not a calculation, but it gives an agreed order for the work.

# Design review action tracker: rank review comments by risk and track closure before the final report.
actions = [  # (id, finding, severity 1-5, likelihood 1-5, owner, status)
    ("DR-01", "ONB margin depends on F_q = 1.9; no pin-power map yet", 4, 3, "Kofi", "open"),
    ("DR-02", "Pool liner leak detection not shown on drawings", 3, 3, "Sofia", "closed"),
    ("DR-03", "Seismic loads on the chimney and core support not assessed", 5, 2, "Mira", "open"),
    ("DR-04", "Fire-water makeup connection needs a credited operator action time", 3, 2, "Sofia", "in progress"),
    ("DR-05", "LCOH excludes decommissioning fund contributions", 3, 4, "Nikolai", "in progress"),
    ("DR-06", "Rod drop time assumed, not justified", 4, 2, "Kofi", "closed"),
    ("DR-07", "N-16 hot water layer design is qualitative only", 3, 3, "Mira", "open"),
    ("DR-08", "Security and safeguards interface not described", 4, 3, "Nikolai", "open"),
    ("DR-09", "Requirements R-07 and R-12 have no verification method", 2, 4, "Hana", "closed"),
    ("DR-10", "Decay heat uses full power for three years: state the conservatism", 2, 2, "Sofia", "closed"),
]
def level(score):
    return "HIGH" if score >= 12 else "MEDIUM" if score >= 6 else "LOW"
ranked = sorted(actions, key=lambda a: -a[2] * a[3])
print(f"{'id':6s}{'risk':>5s} {'level':7s}{'owner':9s}{'status':13s}finding")
for i, f, s, l, o, st in ranked:
    print(f"{i:6s}{s * l:5d} {level(s * l):7s}{o:9s}{st:13s}{f}")
open_high = [a for a in actions if a[5] != "closed" and level(a[2] * a[3]) == "HIGH"]
closed = sum(a[5] == "closed" for a in actions)
print(f"closed {closed} of {len(actions)}; open HIGH items: {', '.join(a[0] for a in open_high) or 'none'}")
by_owner = {}
for a in actions:
    if a[5] != "closed": by_owner[a[4]] = by_owner.get(a[4], 0) + 1
print("open items by owner:", ", ".join(f"{k} {v}" for k, v in sorted(by_owner.items())))
print("gate rule: the final report may go to the sponsor when no HIGH item is open or each has a documented disposition")

id     risk level  owner    status       finding
DR-01    12 HIGH   Kofi     open         ONB margin depends on F_q = 1.9; no pin-power map yet
DR-05    12 HIGH   Nikolai  in progress  LCOH excludes decommissioning fund contributions
DR-08    12 HIGH   Nikolai  open         Security and safeguards interface not described
DR-03    10 MEDIUM Mira     open         Seismic loads on the chimney and core support not assessed
DR-02     9 MEDIUM Sofia    closed       Pool liner leak detection not shown on drawings
DR-07     9 MEDIUM Mira     open         N-16 hot water layer design is qualitative only
DR-06     8 MEDIUM Kofi     closed       Rod drop time assumed, not justified
DR-09     8 MEDIUM Hana     closed       Requirements R-07 and R-12 have no verification method
DR-04     6 MEDIUM Sofia    in progress  Fire-water makeup connection needs a credited operator action time
DR-10     4 LOW    Sofia    closed       Decay heat uses full power for three years: state the conservatism
closed 4 of 10; open HIGH items: DR-01, DR-05, DR-08
open items by owner: Kofi 1, Mira 2, Nikolai 2, Sofia 1
gate rule: the final report may go to the sponsor when no HIGH item is open or each has a documented disposition

The tracker shows three high-risk findings, DR-01, DR-05 and DR-08, two of which are fully open. The board sets a rule for the gate: the team may proceed to the final report when no high-risk finding is open, or when each open high-risk finding has a documented disposition that the board accepts. A disposition can be a resolution, a justified decision that no change is needed, or a deferral to a later phase with a statement of what will be done and why the deferral is acceptable. What is not acceptable is a finding that disappears without a record.

Closing the findings

Over the next two weeks, the team works through the findings in order of risk. For DR-01, Kofi builds a simple two-dimensional model of the core with each assembly represented by its average power and an intra-assembly peaking factor from the published lattice calculation. It gives a total peaking factor of 1.95, a little above the assumed 1.9. Mira reruns the hot channel calculation and finds a boiling margin of about 11 K, still above the 10 K requirement. The team updates the design values, records that the margin is thinner than the first estimate suggested, and adds a requirement for a detailed pin-power calculation in the next phase. The board accepts the disposition and the finding is closed.

For DR-05, Nikolai adds a decommissioning fund contribution to the cost of heat. Using an analogy with decommissioning estimates for pool-type research reactors, scaled for the larger building, he estimates the decommissioning cost and the annual contribution needed to accumulate it over 40 years at a modest real return. The contribution adds roughly 10 to 15 dollars per MWh to the cost of heat, which does not change the conclusion but makes the estimate complete. For DR-08, the team cannot produce a security design in two weeks, and it would not be appropriate for a student team to publish one. Instead it adds a section describing the security and safeguards requirements that will apply, the design features that support them, such as the below-grade pool, the protected area boundary and the separation of digital safety systems from networks, and a commitment that the next phase will include a security assessment by qualified specialists. The board accepts this as a documented deferral.

The medium findings are handled in the same way. DR-03, the seismic loads on the chimney and core support, is deferred with a statement of the analysis needed and an argument that the design can accommodate it: the chimney is a light structure that can be braced to the pool wall, and the control rods are guided within the core structure rather than the chimney. DR-07, the warm surface layer, is partly addressed with a simple calculation of the heater power needed to maintain a 3 K layer against mixing, and a commitment to instrument the layer with temperature sensors at several depths that will alarm if it breaks down. DR-04, the operator action time for the fire-water connection, is resolved by noting that the pool takes about twenty days to boil, so a credited action time of 24 hours has a large margin.

The human side of review

Being reviewed is uncomfortable, and the way a team responds to criticism tells reviewers a great deal. Defensive responses, such as arguing that a finding is out of scope, that the reviewer has misunderstood, or that there was no time, make reviewers more skeptical and less willing to help. Open responses, such as acknowledging the point, explaining what the team knows and does not know, and proposing how to resolve it, build trust. Lise tells the team afterwards that its response to DR-01 was exactly right: Kofi said that the peaking factor was a judgment, explained how he had made it, and offered to check it. That is what a reviewer wants to hear from a designer.

Reviewers have responsibilities too. A good reviewer focuses on the design rather than the designer, explains why a concern matters, distinguishes between what is wrong and what is merely different from how the reviewer would have done it, and accepts a well-argued response. In nuclear organizations, a strong review culture, in which people at every level expect their work to be questioned and are comfortable questioning others, is part of the safety culture that regulators assess. The team's charter, with its questioning attitude, was written with that in mind.

Configuration control after review

After the review, the design is baselined: the reviewed versions of the requirements, design description and calculation packages are frozen, and any further change goes through a change process that checks its effects on other areas before it is accepted. For a student team two weeks from the end, this mainly prevents last-minute changes from creeping into the final report without being reflected everywhere they matter. When Mira suggests a small increase in the heat exchanger area to recover some of the campus supply temperature margin, Hana logs it as a proposed change, checks the traceability matrix for its effects on cost, layout and R-01, and the team approves it with the cost estimate updated to match.