Concept generation and trade studies

With a requirements baseline in hand, the TrailCount team faced the most consequential decision of the project: what kind of system to build. Every later choice, from the circuit board to the test plan, depends on it. Teams that jump straight to the first idea that comes to mind usually end up defending it rather than evaluating it. This lesson shows how to generate genuinely different concepts, how to compare them fairly with a weighted decision matrix and supporting analysis, and how to record the decision so that it survives scrutiny at the preliminary design review.

Divergence before convergence

Concept generation has two phases that should not be mixed. In the divergent phase, the team produces many options without judging them. In the convergent phase, it narrows them down with evidence. Judging too early kills ideas before they are understood, and it also anchors the team on the first concept. Professor Osei asked the team to produce at least eight concepts before scoring any of them.

The most useful divergent tool for a system like TrailCount is functional decomposition followed by a morphological chart. First break the system into functions that any solution must perform, without saying how: sense a passing person, distinguish walking from cycling, determine direction, aggregate counts, store and forward data, harvest or receive energy, store energy, protect the electronics, and mount to a post. Then, for each function, list several ways of doing it. A concept is one path through the chart, choosing one option per row.

Even this partial chart contains 4 × 4 × 4 × 4 × 4 × 4 = 4,096 combinations. Most are nonsense, such as a camera powered by a supercapacitor, but the chart forces the team to consider options it would never have listed otherwise. Keisha had not thought of supercapacitors until she filled in the energy storage row; they were rejected later for their energy density, but they reappeared as a small buffer for radio bursts. Other divergent methods include brainstorming with a rule against criticism, studying existing products and patents, and analogy: Amira pointed out that automatic door openers already solve the problem of detecting approaching people at low power.

The team assembled eight plausible concepts from the chart and screened them against a few pass or fail criteria before any detailed scoring. A concept that clearly violates a hard constraint does not deserve a row in the decision matrix. Pressure mats failed screening because they need excavation of the trail, which the city refused. Wind power failed because the site is sheltered by trees. Manual download failed requirement SYS-05 outright. Four concepts remained.

• Concept A: reuse the NS-5 revision B board, which already has a radar front end and an SoC FPGA, on grid power brought from a streetlight.
• Concept B: a commercial low-power radar module that performs detection on its own processor, a low-power microcontroller for tracking and classification, LoRaWAN, and a solar panel with a LiFePO4 battery.
• Concept C: two passive infrared sensors a known distance apart, a microcontroller, cellular LTE-M, and solar power.
• Concept D: a small camera with an on-board neural network accelerator that classifies images and keeps only counts, with LTE-M and solar power.

Choosing criteria and weights

A decision matrix compares concepts against criteria. The criteria come from the requirements and from project realities. They should be independent of each other, few enough to reason about (five to eight is typical), and stated so that higher scores are always better. The TrailCount team chose six: count accuracy (from SYS-01, SYS-02 and SYS-12), winter energy (SYS-03 and SYS-04), privacy (SYS-06), unit cost (SYS-09), team risk (how likely the team is to deliver a working system with its skills in the time available), and maintenance (site visits over the life of the node).

Weights express relative importance and should sum to 1. The team set them before scoring any concept, to avoid unconsciously choosing weights that favor a preferred design. Each member proposed weights privately, the team discussed the differences, and Ines reviewed the result. Count accuracy received 0.25 because the city said a counter that cannot separate modes is useless. Winter energy received 0.20 because the site has no power. Privacy, cost and team risk each received 0.15, and maintenance 0.10.

Scoring with evidence

Scores are where decision matrices go wrong. A score of 4 for count accuracy means nothing unless the team can say why. The TrailCount team wrote a one-line justification for every score and attached evidence: datasheet figures, a quick experiment, published studies or a calculation. Scores ran from 1 (poor) to 5 (excellent), with anchors defined in advance. For winter energy, for example, 5 meant a concept that would run all winter on a panel smaller than 10 W, and 1 meant one that needed grid power the site does not have.

Several scores came from small experiments rather than opinion. Jonah borrowed a radar evaluation kit and recorded passages at the trail for two afternoons; the radial velocity readings separated walkers (about 3 to 7 km/h) from most cyclists (about 12 to 30 km/h), with runners and slow cyclists overlapping. Amira tested a pair of passive infrared sensors on a campus path and found that they missed people walking side by side and became unreliable on hot afternoons, when people and background were at similar temperatures. Those results drove the accuracy scores of 4 for the radar concept and 2 for the infrared concept.

Concept A scored 5 for accuracy because the NS-5 radar is excellent, but 1 for winter energy and 1 for cost: bringing grid power to the site would cost far more than the node itself, and the NS-5 board draws watts, not milliwatts. Concept D scored 5 for accuracy, but 1 for privacy. Even though the camera would keep only counts, the city's privacy officer said that the presence of a camera on a trail would generate complaints and require an assessment, and the team could not verify within its schedule that no image could ever be extracted.

# Weighted decision matrix with a weight-sensitivity check
criteria = {'Count accuracy': 0.25, 'Winter energy': 0.20, 'Privacy': 0.15,
            'Unit cost': 0.15, 'Team risk': 0.15, 'Maintenance': 0.10}
# scores 1 (poor) to 5 (excellent), agreed by the team with evidence for each
concepts = {
    'A  NS-5 board on grid power':   [5, 1, 5, 1, 4, 3],
    'B  Radar module, LoRaWAN, solar': [4, 4, 5, 4, 3, 4],
    'C  Passive IR pair, LTE-M':     [2, 4, 5, 5, 4, 3],
    'D  Camera with on-board AI':    [5, 2, 1, 3, 2, 3],
}
assert abs(sum(criteria.values()) - 1) < 1e-9
w = list(criteria.values())
def totals(weights):
    return {c: sum(wi * si for wi, si in zip(weights, s)) for c, s in concepts.items()}
base = totals(w)
for c, t in sorted(base.items(), key=lambda x: -x[1]):
    print(f"{c:33s} {t:4.2f}")
best = max(base, key=base.get)

# sensitivity: raise or lower each weight by 50 %, renormalize, and see whether the winner changes
print()
flips = 0
for i, name in enumerate(criteria):
    for f in (0.5, 1.5):
        ww = w[:]; ww[i] *= f; s = sum(ww); ww = [x / s for x in ww]
        t = totals(ww); win = max(t, key=t.get)
        if win != best:
            flips += 1
            print(f"{name} x{f}: winner becomes {win.split()[0]}")
print(f"Winner {best.split()[0]} holds in {12 - flips} of 12 weight changes")
margin = sorted(base.values())[-1] - sorted(base.values())[-2]
print(f"Margin over second place: {margin:.2f} points")

B  Radar module, LoRaWAN, solar   4.00
C  Passive IR pair, LTE-M         3.70
A  NS-5 board on grid power       3.25
D  Camera with on-board AI        2.85

Winner B holds in 12 of 12 weight changes
Margin over second place: 0.30 points

Concept B scored 4.00, concept C 3.70, concept A 3.25 and concept D 2.85. The stacked bars show why: concept B has no weak criterion, while every other concept has at least one score of 1 or 2 on a heavily weighted criterion. That pattern, a winner without a fatal weakness, is more convincing than a large total margin.

Is the result robust?

A margin of 0.30 points out of 5 is not large. If small changes in the weights would change the winner, the decision rests on judgments the team cannot defend precisely, and the team should either gather better evidence or consider both concepts further. The second part of the script multiplies each weight in turn by 0.5 and by 1.5, renormalizes so that the weights still sum to 1, and recomputes the winner. Concept B wins in all twelve cases. The team can tell reviewers that no plausible disagreement about the weights changes the decision.

Score uncertainty matters as much as weight uncertainty. The closest challenger, concept C, would overtake concept B only if its accuracy score rose from 2 to about 3.2 while every other score stayed the same. Amira's experiment showed exactly the failure modes that keep its score low, so the team was comfortable. If the experiment had been inconclusive, the right response would have been another experiment, not more debate.

Beyond the matrix

A decision matrix is a communication tool and a check on reasoning, not an oracle. It hides interactions between criteria, it treats scores as if they were precise, and it adds up strengths and weaknesses that may not be interchangeable. A concept that scores 1 on safety is not rescued by scoring 5 on cost. For that reason, serious trade studies combine the matrix with three other tools.

• Hard constraints, applied as screening before scoring, as the team did with pressure mats and manual download.
• Quantitative analysis of the criteria that matter most. For winter energy, Keisha built the power budget that appears in lesson 4 for each concept rather than relying on intuition. For cost, Lucas priced each concept's main parts at a quantity of 100.
• Risk assessment of the top concepts: what could stop each one from working, and how quickly could the team find out? Concept B's biggest risk was classification accuracy; concept C's was heat and side-by-side walkers. A cheap experiment for each top risk is often worth more than refining scores.

Engineers also use Pugh's method, an earlier and simpler technique in which each concept is compared with a reference concept and marked better, same or worse on each criterion. It is fast and good for early screening, and it avoids false precision. Many teams use a Pugh comparison to narrow eight concepts to three or four, then a weighted matrix and analysis to choose among the finalists.

Recording the decision

The team wrote a two-page trade study memo: the concepts, the screening, the criteria and weights with their rationale, the scores with evidence, the totals, the sensitivity analysis, and the decision with the main risks carried forward. It went into the decision log and into an appendix of the final report. When the city's privacy officer later asked, in a public meeting, whether the team had considered cameras, Jonah could answer in one sentence and offer the memo.

Recording the rejected alternatives also protects the client. A year later, a new Northway engineer who wonders why the product does not reuse the NS-5 board can read the reasons, including the cost of bringing grid power to a trail, instead of repeating the study.