Software and hardware assurance (DO-178C and DO-254)

Almost every system in this course is controlled by software. The Tern Block 2's battery management units decide when the contactors may close; its propulsion controllers command the motors; its displays show the crew how much energy remains. The Swift 900 study's flight control computers run the pitch law of AERO-330 and its FADEC engine controllers meter the fuel. A software error in any of these could be as dangerous as a broken spar, yet software does not wear out or fail at random. It does exactly what it was written to do, every time, including the wrong thing. This final lesson explains how the industry gains confidence that airborne software and complex electronic hardware are correct, through RTCA DO-178C and DO-254, and then brings the course together in the Tern's systems certification plan.

Why software needs a different approach

Lesson 1 showed how fault trees give probabilities for random hardware failures. That approach does not work for software. A design error is present in every copy; whether it causes harm depends on whether a particular combination of inputs ever occurs, and the number of possible combinations in even a small program is far too large to test exhaustively. Nobody can show by testing alone that a flight control program will never misbehave in 10⁹ hours. Instead, the industry relies on development assurance: a disciplined process, scaled to the consequences of failure, that makes errors unlikely to be introduced and likely to be found if they are.

DO-178C, Software Considerations in Airborne Systems and Equipment Certification, published in 2011, is the standard for this process. In the United States the FAA recognises it as an acceptable means of compliance in AC 20-115D, issued in 2017; Transport Canada accepts it through its own advisory material, and most other authorities do the same. Four supplements extend it: DO-330 on qualifying software tools, DO-331 on model-based development, DO-332 on object-oriented technology and DO-333 on formal methods.

Software levels and objectives

DO-178C assigns each software component a level from A to E, following the development assurance level of the function it implements, which in turn follows the failure condition classification of Lesson 1. It then lists objectives that the development must satisfy: plans, standards, requirements, design, code, integration, verification, configuration management, quality assurance and certification liaison. The higher the level, the more objectives apply, and the more of them must be satisfied with independence, meaning that the person verifying an item is not the person who produced it.

# Software assurance: DO-178C objective counts by level, and a minimal set of MC/DC test cases for
# the Tern battery contactor close logic, with a check that each condition independently matters.
from itertools import product
objectives = {'A': (71, 30), 'B': (69, 18), 'C': (62, 5), 'D': (26, 2), 'E': (0, 0)}
coverage = {'A': 'MC/DC + decision + statement', 'B': 'decision + statement', 'C': 'statement', 'D': 'none required', 'E': '-'}
print("level  objectives  with independence  structural coverage")
for lv, (n, ind) in objectives.items():
    print(f"  {lv}       {n:3d}           {ind:3d}            {coverage[lv]}")
# Contactor may close only if: precharge is complete AND (no insulation fault OR maintenance override)
def close(pre, nofault, override): return pre and (nofault or override)
names = ('precharge complete', 'no insulation fault', 'maintenance override')
cases = list(product((False, True), repeat=3))
print(f"\nall {len(cases)} input combinations; searching for an MC/DC set")
def shows(i, a, b):          # pair a, b shows condition i independently affects the outcome
    return all(a[j] == b[j] for j in range(3) if j != i) and a[i] != b[i] and close(*a) != close(*b)
best = None
from itertools import combinations
for k in range(2, 9):
    for s in combinations(cases, k):
        if all(any(shows(i, a, b) for a in s for b in s) for i in range(3)):
            best = s; break
    if best: break
print(f"minimal MC/DC set has {len(best)} cases (n + 1 = {len(names)+1}):")
for c in best:
    print("   " + ", ".join(f"{n}={'T' if v else 'F'}" for n, v in zip(names, c)) + f"  -> close={close(*c)}")
# Statement coverage from a test run: lines executed out of lines in the module
executed, total = 1838, 1912
print(f"\nstatement coverage {100*executed/total:.1f} %: the {total-executed} lines not executed must be "
      f"explained (dead code removed, deactivated code justified) or covered by more requirement-based tests")

level  objectives  with independence  structural coverage
  A        71            30            MC/DC + decision + statement
  B        69            18            decision + statement
  C        62             5            statement
  D        26             2            none required
  E         0             0            -

all 8 input combinations; searching for an MC/DC set
minimal MC/DC set has 4 cases (n + 1 = 4):
   precharge complete=F, no insulation fault=F, maintenance override=T  -> close=False
   precharge complete=T, no insulation fault=F, maintenance override=F  -> close=False
   precharge complete=T, no insulation fault=F, maintenance override=T  -> close=True
   precharge complete=T, no insulation fault=T, maintenance override=F  -> close=True

statement coverage 96.1 %: the 74 lines not executed must be explained (dead code removed, deactivated code justified) or covered by more requirement-based tests

The first table in the program summarises the standard: 71 objectives at level A, 30 of them with independence; 69 at level B with 18; 62 at level C with 5; 26 at level D with 2; and none at level E. The difference between levels is not only the count. The verification objectives change most: level A demands the most thorough testing and analysis, and structural coverage of the code by the requirement-based tests, explained below, becomes more demanding as the level rises.

Requirements, traceability and testing

The core of DO-178C is the chain from system requirements, allocated to software by the ARP4754B process of Lesson 1, to high-level software requirements, to low-level requirements and design, to source code and finally to executable object code. Every item must trace to its parent and to the verification that shows it is correct. Code that does not trace to a requirement is either dead code, which must be removed, or deactivated code, which must be shown to be safely disabled in the aircraft's configuration. Requirements must be accurate, consistent, verifiable and free of implementation detail at the higher levels, and reviews check them against these criteria.

Testing in DO-178C is requirement-based: tests are written from the requirements, not from the code, so that they check whether the software does what it should rather than merely what it happens to do. Normal range tests check expected behaviour; robustness tests check behaviour with abnormal inputs such as out-of-range values, failed sensors and timing faults. Testing is performed on the target hardware or a representative environment, because compilers and processors can introduce their own problems.

Structural coverage and MC/DC

Requirement-based tests might still leave parts of the code untested, if the requirements are incomplete or the code contains something the requirements do not describe. Structural coverage analysis measures which parts of the code the requirement-based tests actually exercised. Statement coverage means every statement was executed; decision coverage means every decision took both true and false outcomes; modified condition/decision coverage, MC/DC, required at level A, means each condition within a decision was shown to affect the decision's outcome independently.

The program illustrates MC/DC on a piece of the Tern's battery management logic: a contactor may close only if precharge is complete and either there is no insulation fault or a maintenance override is active. The decision has three conditions and eight possible input combinations. MC/DC requires pairs of tests in which only one condition changes and the outcome changes with it, for each condition. The program searches for the smallest set and finds four cases, the minimum of n + 1 for n conditions. Exhaustive testing would need eight here, but for a decision with ten conditions it would need 1024 while MC/DC needs as few as eleven, which is why MC/DC is practical for real software.

Coverage gaps are findings, not failures. The program's last line reports 96.1 percent statement coverage on a module, with 74 lines not executed. Each gap must be resolved: by writing a missing requirement and its test, by removing dead code, or by justifying deactivated code. Arjun found that 50 of the 74 lines were a diagnostic routine used only on the systems rig, which became a documented deactivated function, and the rest revealed a missing requirement for handling a stale temperature reading, which was added and tested.

Tools, models and formal methods

Tools that generate code or automate verification can introduce errors or hide them. DO-330 sets how a tool must be qualified, with more rigour for a tool whose output goes into the aircraft, such as a code generator, than for one that only helps verification. Model-based development under DO-331 lets engineers design control laws as block diagrams and generate code from them, common for flight controls; the model itself becomes a requirement or design artefact with its own verification. Formal methods under DO-333 use mathematical proof to show that software has certain properties, for example that a monitor can never command contradictory outputs, and can replace some testing.

Complex electronic hardware: DO-254

Functions are increasingly implemented in programmable logic such as field-programmable gate arrays rather than in software. Such complex electronic hardware has the same problem as software: design errors cannot be found by testing every case. RTCA DO-254, Design Assurance Guidance for Airborne Electronic Hardware, provides the equivalent process: planning, requirements capture, conceptual and detailed design, implementation, verification and configuration management, with rigour by design assurance level. The FAA's AC 20-152A, issued in 2022, explains how DO-254 is applied to custom microcoded components such as FPGAs. Simple hardware, whose behaviour can be fully verified by test and analysis, does not need this process.

The Tern's battery management units and propulsion controllers both use FPGAs for fast protection functions, such as opening a contactor within microseconds of a short circuit. These are developed to DO-254 at the same assurance level as the software they work with. Commercial processors used in avionics bring their own concern, since their internal design is not visible; guidance requires the applicant to assess and mitigate their errata and features, especially for multi-core processors where cores share memory and buses and can interfere with each other's timing.

Capstone: the Tern Block 2 systems certification plan

Élise Fournier's final task for the systems team brings the course together. They must present the Tern Block 2's systems certification plan to Transport Canada. The plan begins with the certification basis and special conditions of Lesson 1 and the functional hazard assessment. The team's table assigns assurance levels to each function: propulsion control and the battery contactor logic at level A, because total loss of propulsive power is catastrophic; the energy display at level B, because misleading state of charge is hazardous; the windshield heat controller at level C; cabin lighting at level E.

For the two battery management units, the team has chosen dissimilar designs, one built in house and one from a supplier, which under ARP4754B allows each to be developed at level B while the combination meets the level A function, provided the independence is demonstrated. The plan lists the evidence: separate requirements teams, different processors, different compilers and different tool chains, and a common cause analysis showing no shared component whose failure could affect both. The plan also records the decisions taken earlier in the course: the split battery strings and separate harness routes, the converter load shedding of Lesson 2, the electric actuation of Lesson 3, the zonal separation of high-voltage cables from fuel in Lesson 4, the propulsion waste heat used for cabin heating in Lesson 5, the boots and heated propellers of Lesson 6, the electric brakes and gravity extension of Lesson 7, the dual sensor architecture of Lesson 8, the coupled RNP procedures of Lesson 9, the space-based ADS-B antennas of Lesson 10 and the energy page of Lesson 11.

The regulator's specialists ask hard questions, as they should. How will the team show that the two battery management units' requirements, which both come from the same system specification, do not share a common error? The answer is that the system requirements themselves are developed and verified at level A, so the shared source is covered by the most rigorous process. How will the team handle a software change after certification? Through a change impact analysis that traces the change through the requirements and decides what must be re-verified. At the end of the meeting the plan is accepted with actions, which is the usual outcome. Élise tells the team that this is what systems engineering looks like when it works: not a single clever design, but a chain of decisions, each traceable to a reason, that together make the aircraft safe enough to fly people.