Risk management and failure modes and effects analysis

Every capstone project faces two kinds of risk. Project risks threaten the team's ability to deliver: a part on backorder, a teammate who falls ill, a supplier who stops answering email. Product risks threaten the system's ability to work safely and reliably once delivered: a battery that swells in the cold, a gasket that leaks, a firmware update that bricks a node. Experienced engineers manage both deliberately, because hope is not a plan. This lesson covers the risk register for project and technical risks, and failure modes and effects analysis (FMEA) for the product.

What a risk is

A risk is an uncertain event or condition that, if it occurs, affects an objective. It has two dimensions: likelihood (how probable it is) and consequence (how bad it would be). A risk is not an issue. An issue has already happened and needs a fix; a risk might happen and needs a plan. Good risk statements follow a cause, event and effect form: because the radar module has a single supplier with a 12-week lead time (cause), the module may arrive after the alpha build date (event), which would delay integration by up to four weeks (effect). That form makes the risk specific enough to act on.

The risk management cycle

Standards such as ISO 31000 describe risk management as a continuing cycle rather than a one-time exercise. For a capstone team, the practical version has five steps, repeated at every weekly meeting and in depth at each design review.

• Identify: brainstorm risks from the requirements, the architecture, the schedule, the team and past projects. Professor Osei gave the team a list of the most common capstone risks from previous years; nearly all of them applied.
• Analyze: estimate likelihood and consequence, each on a scale of 1 to 5 with defined anchors, and place the risk on a matrix.
• Plan responses: decide how to treat each significant risk.
• Implement: assign each response an owner and a date, and do it.
• Monitor: review the register weekly, update scores, close risks that have passed, and add new ones.

There are four classic responses. Avoid the risk by changing the plan so it cannot occur, as the team did by buying a radar module instead of designing a radar. Mitigate it by reducing its likelihood or consequence, such as ordering two modules from two distributors. Transfer it to someone better able to handle it, such as having Northway's network team own gateway coverage. Accept it, with a contingency plan if it occurs, when the cost of treatment exceeds the expected harm.

The TrailCount risk register

The register is a living document. At the preliminary design review, R1 sat in a red cell. By week 12 both radar modules had arrived, and the risk was closed. R3 rose in week 15, when the first field recordings showed runners being classified as cyclists, and fell again in week 19 after Amira added a speed-spread feature. Watching scores move tells the team, and its reviewers, where attention is needed.

Failure modes and effects analysis

FMEA is a structured, bottom-up method for finding how a product can fail and what to do about it before it fails in the field. It was developed for military and aerospace systems and is now standard in automotive, medical and industrial products. A design FMEA examines the product's design; a process FMEA examines how it is made. For TrailCount the team did a design FMEA.

For each item, such as a component, subsystem or function, the team asks: how could it fail to do its job (failure mode)? What would happen (effect)? Why would it happen (cause)? What in the current design prevents the cause or detects the failure? Then it rates three things on scales of 1 to 10, using tables agreed in advance.

• Severity (S) of the effect: 1 for no noticeable effect, up to 9 or 10 for effects involving safety or regulatory non-compliance.
• Occurrence (O) of the cause: 1 for extremely unlikely, up to 10 for almost certain, given the current prevention controls.
• Detection (D): how likely the current controls are to detect the cause or failure before the product reaches the user. Here the scale is reversed: 1 means almost certain detection, 10 means no chance of detection.

The traditional summary is the risk priority number, the product of the three ratings: RPN = S × O × D, ranging from 1 to 1,000. Teams sort by RPN and act on the highest.

# Design FMEA for the TrailCount node: rank by RPN, then apply a severity-first rule
fmea = [  # item, failure mode, effect, S, O, D
    ('Battery charger', 'Charges LiFePO4 below 0 C', 'Lithium plating, capacity loss, swelling', 9, 6, 4),
    ('Radar firmware', 'Two walkers side by side counted as one', 'Undercount on busy days', 5, 7, 5),
    ('Enclosure seal', 'Gasket leaks at cable gland', 'Corrosion, node stops', 7, 4, 6),
    ('Solar input', 'Panel covered by snow for days', 'Battery drains, data gap', 6, 6, 2),
    ('LoRa link', 'Gateway out of range after leaves grow', 'Counts held, sent late', 4, 4, 3),
    ('Bootloader', 'Power fails during firmware update', 'Node will not start, site visit', 8, 3, 7),
    ('Clock', 'RTC drifts 2 s per day', 'Hourly bins slightly shifted', 3, 8, 3),
    ('Mounting', 'Bracket loosens, radar tilts', 'Missed detections far side', 6, 3, 5),
]
rows = [(s * o * d, s, o, d, item, mode, eff) for item, mode, eff, s, o, d in fmea]
print('By RPN = S x O x D')
for rpn, s, o, d, item, mode, eff in sorted(rows, reverse=True):
    print(f"{rpn:4d}  S{s} O{o} D{d}  {item}: {mode}")

def priority(s, o, d):
    """A simple severity-first rule in the spirit of action priority tables."""
    if s >= 9 or (s >= 7 and o >= 4) or (s >= 7 and d >= 7): return 'High'
    if s >= 5 and (o >= 5 or d >= 6): return 'Medium'
    return 'Low'
print('\nSeverity-first priority')
for rpn, s, o, d, item, mode, eff in sorted(rows, key=lambda r: ('High', 'Medium', 'Low').index(priority(*r[1:4]))):
    print(f"{priority(s, o, d):6s}  RPN {rpn:3d}  {item}: {mode}")

By RPN = S x O x D
 216  S9 O6 D4  Battery charger: Charges LiFePO4 below 0 C
 175  S5 O7 D5  Radar firmware: Two walkers side by side counted as one
 168  S8 O3 D7  Bootloader: Power fails during firmware update
 168  S7 O4 D6  Enclosure seal: Gasket leaks at cable gland
  90  S6 O3 D5  Mounting: Bracket loosens, radar tilts
  72  S6 O6 D2  Solar input: Panel covered by snow for days
  72  S3 O8 D3  Clock: RTC drifts 2 s per day
  48  S4 O4 D3  LoRa link: Gateway out of range after leaves grow

Severity-first priority
High    RPN 216  Battery charger: Charges LiFePO4 below 0 C
High    RPN 168  Enclosure seal: Gasket leaks at cable gland
High    RPN 168  Bootloader: Power fails during firmware update
Medium  RPN 175  Radar firmware: Two walkers side by side counted as one
Medium  RPN  72  Solar input: Panel covered by snow for days
Low     RPN  48  LoRa link: Gateway out of range after leaves grow
Low     RPN  72  Clock: RTC drifts 2 s per day
Low     RPN  90  Mounting: Bracket loosens, radar tilts

Why RPN alone is not enough

Sorting by RPN put the cold-charging failure first with 216, which nobody disputed. But second place went to the side-by-side walkers, with 175, ahead of the bootloader and enclosure failures at 168. Is a modest undercount on busy days really a higher priority than a node that will not start after a failed firmware update and needs a site visit, or a gasket that lets water in and kills the electronics? Most engineers would say no.

RPN has well-known weaknesses. Multiplying ordinal scores is mathematically dubious: a 3 on one scale is not three times a 1. Very different risks produce the same RPN, as the clock drift and snow on the panel both score 72. And a high-severity failure can hide behind a low RPN if it is rare and easy to detect. For these reasons, the AIAG and VDA FMEA handbook published in 2019 for the automotive industry replaced RPN with action priority, a lookup table that gives severity the most weight, then occurrence, then detection, and assigns each combination a high, medium or low priority.

The script's second ranking applies a simple severity-first rule in that spirit. It is not the handbook's table, which is larger and more detailed, but it captures the idea: anything with severity 9 or above is high priority regardless of the other scores, and severity 7 or above combined with moderate occurrence or poor detection is also high. Under this rule the enclosure seal and the bootloader join the cold-charging failure at the top, and the side-by-side walkers move to medium. The team agreed that this ordering matched its engineering judgment, and adopted it.

The example illustrates a principle that the handbooks emphasize: the order of preference for actions is to reduce severity first (by design changes that make the effect less harmful), then occurrence (by preventing the cause), and only then detection (by better testing). Detection alone never makes a product safer in the field; it only catches problems before shipment.

Acting on the TrailCount FMEA

The three high-priority items each led to design changes, owners and verification steps, recorded in the FMEA's action columns.

• Cold charging (S9 O6 D4): a charger with temperature inputs that blocks charging below 0 °C, a 2 W heater pad on the cell controlled by firmware and powered only from panel surplus, an insulated battery compartment, and a cold-chamber test at −30 °C with the panel simulated by a bench supply. Severity cannot fall, because damage to a lithium cell is a safety matter, but occurrence falls to 2 and detection to 3.
• Enclosure seal (S7 O4 D6): a cable gland rated for the cable's diameter, a drip loop, a breathable vent membrane to stop pressure changes from pumping moist air through the seals, and an ingress test to IP66 on the beta enclosure before the field trial.
• Bootloader (S8 O3 D7): the dual-bank update design in the worked example, with signature verification of each image, discussed further in lesson 10.

Medium-priority items received actions too, but with later dates. The side-by-side walkers became a firmware improvement in the tracker and a specific test case in the verification plan. Snow on the panel led to a steeper mounting angle, which sheds snow and improves winter harvest, and to a status flag that shows a solar deficit on the dashboard.

FMEA and the rest of the project

FMEA connects to almost everything else in the course. Its failure modes become test cases in the verification plan. Its causes become derived requirements, such as the charger temperature cutoff. Its detection controls become manufacturing tests for the pilot units. And its severity ratings feed the safety analysis in lesson 10. A related top-down method, fault tree analysis, starts from an undesired top event, such as the node stops reporting, and works down through logical AND and OR gates to the basic causes. FMEA finds failures you did not think to ask about; fault trees show how combinations of failures produce the outcome you care about most. Larger projects use both.

An FMEA is only as good as the people in the room. The team held two sessions of about two hours each, with all four members, Professor Osei for the first half hour and, for the power section, a Northway technician who had serviced outdoor equipment for years. The technician added two failure modes the team had not considered: spiders nesting in vent openings, and installers overtightening the cable gland and cracking it.