Storage, I/O and a capstone design trade study

A processor is useful only if it can exchange data with the world. Sensors deliver samples, radios carry messages, and storage keeps results for later. The input and output system decides how much processor time and energy that exchange costs, and storage devices have performance and lifetime properties as important to a design as any cache parameter. This final lesson covers flash storage, the three ways a processor can manage input and output, and the buses and interconnects that join the parts of a system. It then brings the whole course together in the decision Maren has been building toward: which processor configuration the NS-5 should use.

The NS-5 receives 2,048 radar samples every 50 ms from an analogue-to-digital converter, writes classification results and selected raw frames to a removable flash card for the data science team, and sends hourly summaries over its cellular radio. Every one of these paths affects the choice of core, because a design that spends its cycles moving data has fewer left for the classifier and burns energy the solar panel must replace.

Flash storage

Flash memory stores charge in floating-gate or charge-trap transistors and keeps it without power. NAND flash, used in solid-state drives, memory cards and embedded storage, is organized into pages of a few kilobytes, which are the unit of reading and writing, grouped into erase blocks of hundreds of pages. A page can be written only after its whole block has been erased, and each block survives a limited number of program and erase cycles, from about 100,000 for single-level cells storing one bit per cell down to a few thousand or fewer for cells storing three or four bits. Reads take tens of microseconds, writes hundreds of microseconds and erases milliseconds: far slower than DRAM but far faster than a rotating disk, whose mechanical seek and rotation cost several milliseconds per random access.

Because pages cannot be overwritten in place, a flash device contains a controller running a flash translation layer, which maps logical block addresses to physical pages, much as a page table maps virtual to physical addresses. A write goes to a fresh, already-erased page, and the old copy is marked invalid. Periodically, garbage collection picks a block with many invalid pages, copies its remaining valid pages elsewhere and erases the block. Those extra copies mean that the flash performs more physical writes than the host requested. The ratio is the write amplification factor, and it multiplies wear. Small random writes and a nearly full device produce high write amplification; large sequential writes, free space reserved by the controller as over-provisioning, and the trim command, by which the file system tells the device which data are no longer needed, keep it low. Wear levelling spreads erases evenly so that no block wears out early.

Jonah's measurement on an NS-4 card showed write amplification near 8, because the logging code appended 64-byte records with a file system sync after each one, forcing the controller to rewrite partial pages again and again. Buffering records in RAM and writing whole 16 KB chunks lowered it to about 1.2. The calculation in the program below shows why the change matters for a node expected to last ten years on a pole.

Polling, interrupts and DMA

Processors talk to devices through memory-mapped registers: a device's status, control and data registers occupy physical addresses, and ordinary loads and stores reach them. These addresses must not be cached, and their accesses must not be reordered, so the memory system marks them as device memory and the RISC-V fence instruction orders them where needed. Three methods decide when the processor moves data.

With polling, the processor repeatedly reads a status register until the device is ready, then transfers the data. Polling is simple and has the lowest latency when the device is fast and always busy, but a core spinning on a status register does nothing else and cannot sleep. With interrupts, the device signals the processor when it needs attention; the processor saves its state, runs a handler, and returns. Interrupts let the core do other work or sleep, but each one costs tens to hundreds of cycles for entry, exit and the disruption of the pipeline and caches, so an interrupt per byte or per sample is expensive at high data rates. With direct memory access, a DMA controller moves blocks of data between the device and memory on its own, and interrupts the processor only when a whole buffer is complete. The processor sets up a descriptor naming the source, destination and length, then is free until the transfer ends.

DMA raises its own architectural questions. Because the DMA engine writes memory directly, the processor's caches may hold stale copies of the buffer, so either the DMA traffic must be coherent, snooped by the caches as in lesson 10, or software must flush and invalidate the buffer's cache lines around each transfer. On systems with virtual memory, an input-output memory management unit translates device addresses and restricts each device to its own buffers, which protects the system from faulty or malicious devices in the same way the MMU protects processes from each other.

Buses and interconnect

The components of a system are joined by buses and point-to-point links. On a chip, a bus or a network-on-chip connects cores, caches, memory controllers and peripherals; in small microcontrollers a simple shared bus with an arbiter suffices, while large chips use crossbars, rings or meshes to give many units concurrent paths. Off the chip, serial links have replaced wide parallel buses because at high speeds keeping many parallel wires in step is harder than sending one fast lane per direction; peripheral interconnects for storage and accelerators, memory interfaces, and simple embedded links such as SPI and I2C all follow the same pattern of trading wires for signalling rate. Bus bandwidth is width times transfer rate, less overhead for arbitration, addresses and protocol, and contention among masters adds latency. For the NS-5, the radar converter feeds an SPI port clocked at about 1.3 MHz, the flash card uses a 4-bit interface, and the radio uses a serial port, all on one peripheral bus whose total load is well under its capacity.

Reliability and dependability

Lesson 1 introduced mean time to failure and availability. In storage and memory, dependability also comes from codes. Error-correcting codes in DRAM and caches detect and correct single-bit errors caused by radiation or marginal cells, and flash controllers use strong codes because flash cells grow noisier as they wear. Redundant arrays of disks or drives combine several devices with parity or mirroring so that data survive the failure of one. A pole-mounted sensor cannot use a disk array, but it can protect its flash with checksums on every record, keep two copies of its configuration, and use a watchdog timer that resets the processor if the firmware stops responding.

The NS-5 capstone trade study

The course has given Maren and Jonah the models they need. The trade study compares four configurations: Core A, the single-issue in-order core; Core B, the dual-issue in-order core with packed SIMD; Core B with the integer neural network accelerator from lesson 11; and a dual-core cluster of two Core B processors, one for the radio and network and one for the classifier, as proposed in lesson 10. A good trade study separates hard constraints, which an option must satisfy or be eliminated, from soft criteria, which are weighted and scored. Here the hard constraint is energy: in a Kingston or Ann Arbor December, the solar panel and battery can supply about 12 mW on average to the processor subsystem. The soft criteria are energy margin, classification time, die area, which drives cost and leakage, and software effort.

Classification runs on a batch of radar frames about every 2 s on a busy road. The program computes the CPU cost of the three input methods, the card lifetime at three write amplification factors, each option's energy per batch and average power including sleep power, and the weighted scores of the options that pass the energy constraint. Times and active power come from lessons 1 and 11; sleep power and area are from the vendors' estimates, with area normalized to Core A.

# NS-5 I/O overhead, flash endurance and a weighted trade study
f_cpu = 160e6
samples_per_s = 2048 / 0.050                 # radar ADC: 2,048 samples per 50 ms frame
print("I/O method        CPU cycles/s   share of core")
io = {
    "polling":        f_cpu,                          # core spins on the status register
    "interrupt/sample": samples_per_s * 60,           # 60 cycles per interrupt entry and exit
    "DMA, 1 irq/frame": (1 / 0.050) * (60 + 400),     # one interrupt plus buffer setup per frame
}
for name, cyc in io.items():
    print(f"{name:17s} {cyc:12.0f}   {cyc / f_cpu:8.2%}")

# flash endurance: 512 MB card, 3,000 erase cycles, logging 200 MB of frames per day
cap, pe, host_per_day = 512e6, 3000, 200e6
for wa in (1.2, 3.0, 8.0):
    years = cap * pe / (host_per_day * wa) / 365
    print(f"write amplification {wa:3.1f}: about {years:4.1f} years of logging")

# energy per batch and average power at one batch every 2 s
options = {
    "A":            dict(t=0.336, p=72.6, sleep=0.40, area=1.0, effort=1.0),
    "B":            dict(t=0.270, p=72.0, sleep=0.55, area=1.6, effort=1.0),
    "B + accel":    dict(t=0.270 / 2.58, p=81.0, sleep=0.85, area=2.3, effort=2.0),
    "dual B":       dict(t=0.270, p=72.0, sleep=1.10, area=3.2, effort=1.6),
}
period, budget = 2.0, 12.0   # seconds between batches, mW for the processor in winter
print("option      t(ms)  E(mJ)  avg(mW)  fits")
for name, o in options.items():
    e = o["p"] * o["t"]
    avg = e / period + o["sleep"]
    o["e"], o["avg"] = e, avg
    print(f"{name:10s} {o['t']*1e3:6.1f} {e:6.2f} {avg:8.2f}  {'yes' if avg <= budget else 'no'}")

# weighted score: each criterion normalized so the best option scores 1
weights = dict(energy=0.40, speed=0.25, area=0.20, effort=0.15)
def score(o, weights=weights):
    best = {k: min(x[k] for x in feasible.values()) for k in ("avg", "t", "area", "effort")}
    return (weights["energy"] * best["avg"] / o["avg"] + weights["speed"] * best["t"] / o["t"]
            + weights["area"] * best["area"] / o["area"] + weights["effort"] * best["effort"] / o["effort"])
feasible = {n: o for n, o in options.items() if o["avg"] <= budget}   # hard constraint first
for name, o in sorted(feasible.items(), key=lambda kv: -score(kv[1])):
    print(f"{name:10s} score {score(o):.3f}")
# sensitivity: what if area and software effort matter much more?
alt = dict(energy=0.20, speed=0.10, area=0.35, effort=0.35)
print("with area and effort weighted heavily:")
for name, o in sorted(feasible.items(), key=lambda kv: -score(kv[1], alt)):
    print(f"{name:10s} score {score(o, alt):.3f}")

I/O method        CPU cycles/s   share of core
polling              160000000    100.00%
interrupt/sample       2457600      1.54%
DMA, 1 irq/frame          9200      0.01%
write amplification 1.2: about 17.5 years of logging
write amplification 3.0: about  7.0 years of logging
write amplification 8.0: about  2.6 years of logging
option      t(ms)  E(mJ)  avg(mW)  fits
A           336.0  24.39    12.60  no
B           270.0  19.44    10.27  yes
B + accel   104.7   8.48     5.09  yes
dual B      270.0  19.44    10.82  yes
B + accel  score 0.864
B          score 0.645
dual B     score 0.479
with area and effort weighted heavily:
B          score 0.838
B + accel  score 0.718
dual B     score 0.527

The input results are decisive on their own. Polling would consume the whole core and prevent it from sleeping; an interrupt per sample would cost only 1.5 percent of the core's cycles but would wake it 40,960 times a second, which keeps it out of its deepest sleep states; DMA with one interrupt per frame costs essentially nothing. Every option will therefore use DMA for the radar converter. The flash results show that buffered writing turns a card that would wear out in under three years into one that outlasts the node.

The energy results eliminate Core A. Although it is the smallest and simplest, its 12.6 mW average exceeds the 12 mW winter budget, because its higher energy per batch outweighs its lower sleep power. Among the remaining options, Core B with the accelerator wins with the agreed weights: it uses half the average power of Core B, classifies a batch in about 105 ms instead of 270, and its larger area and software effort cost less than its energy and speed gain. The dual-core cluster scores lowest: it adds the leakage and area of a second core without speeding up the classifier at all, and its main benefit, isolating the radio from the classifier, can be had with priority interrupts and DMA on one core.

A weighted score is only as trustworthy as its weights, so the program repeats the scoring with area and software effort given much more weight. The ranking then flips: plain Core B comes first. This sensitivity analysis tells Maren that the decision really turns on one question, how much the team values energy margin compared with silicon cost and the effort of adopting the accelerator's toolchain. She also checks robustness in other directions. If traffic is heavier and batches arrive every second, Core B's average power rises to about 20 mW and fails the budget, while Core B with the accelerator rises to about 9 mW and still fits. If the classifier grows, as neural networks tend to, the accelerator absorbs most of the growth, while the plain core's energy grows in proportion.

Maren's recommendation to Northway's engineering review is Core B with the integer accelerator, using DMA for all sensor input and buffered, page-aligned flash writes. She documents the reasoning in the form the course has taught: the workload and its measured instruction count, the iron law and Amdahl's law estimates checked against measurement, the energy constraint that eliminated Core A, the scores with their weights, the sensitivity analysis showing what would change the decision, and the risks, chiefly that the accelerator's toolchain must support the classifier's operators. Dana's server team draws its own conclusions from lessons 8 and 10: tiling their aggregation loops and padding per-thread counters cost nothing in hardware and should come before any server purchase.

The broader lesson is that architecture is a discipline of quantified trade-offs. No configuration is best in every dimension. Pipelines, predictors, out-of-order windows, caches, TLBs, coherence protocols, vector units and accelerators each buy performance with area, power and complexity, and the right choice depends on the workload and constraints. An architect's job is to measure the workload, model the options, eliminate those that violate hard limits, weigh the rest honestly, and test how sensitive the answer is to the assumptions.