Capstone: an agent for an invented domain, ethics and the field today

Each lesson so far has isolated one technique. Real agents combine several: they search for routes, reason about what they cannot see, weigh uncertain outcomes and replan when the world surprises them. This capstone builds one such agent for an invented but realistic domain, a sidewalk-clearing robot on a campus after an overnight storm, using A* search, an optimization over task orderings, Bayesian belief updating, expected-utility decisions and the value of information. It then evaluates the whole agent against a fixed routine. The second half of the lesson steps back to ask what responsibilities come with building such systems, how Canada and the United States are approaching AI governance, and where the field stands today.

The task: a sidewalk-clearing robot

Our robot, which we will call Plowbot, starts each storm night at a depot. Five building entrances must be cleared and, if icy, salted: a residence, a lab, a gym and two accessible ramps at the library and the clinic. Plowbot carries a surface sensor that reports ice correctly 85 per cent of the time and gives false alarms 10 per cent of the time. Salting costs material and causes environmental damage to nearby soil and water; leaving ice untreated risks injuries, and the harm is greater at busy or accessible entrances. Following lesson 1, the first step is a PEAS description.

Classified by lesson 1's dimensions, the task is partially observable (ice is hidden), stochastic (the sensor is noisy and the weather varies), sequential (the order of visits matters) and single-agent for planning purposes. No one technique from the course handles all of that, so the agent's design splits the problem into parts that each match a technique, and then connects them. This decomposition is the most important design decision; much of the art of building AI systems lies in it.

Part 1: routes and the order of visits

The campus is a grid of 10-metre sidewalk segments with buildings as obstacles. Plowbot needs the shortest path between every pair of points of interest (the depot and five entrances), which is A* with the Manhattan-distance heuristic from lesson 3, admissible on a four-connected grid. With those 30 distances in hand, it must choose the order of visits. With five entrances there are only 5! = 120 orders, so it can try them all; a campus with 50 entrances would need the local search of lesson 4 or a constraint-based scheduler from lesson 6.

What should the order optimize? The shortest tour is the obvious answer, but it ignores the fact that some entrances matter more and sooner. The program compares the shortest tour with the order that minimizes priority-weighted waiting time: the sum over entrances of priority multiplied by the time at which that entrance is cleared.

# The capstone domain: a sidewalk-clearing robot on an invented campus after an overnight storm.
CAMPUS = [
    "D....#......",
    ".##..#.###..",
    ".##.......#.",
    "....###.#.#.",
    ".#....#.#...",
    ".#.##...###.",
    "...#..#.....",
]  # D = depot, # = building, . = path (each cell is a 10 m sidewalk segment)
H, W = len(CAMPUS), len(CAMPUS[0])
FREE = {(r, c) for r in range(H) for c in range(W) if CAMPUS[r][c] != "#"}
DEPOT = (0, 0)
# Entrances that must be cleared: name, cell, priority weight (accessible entrances weigh more)
TASKS = [("residence", (4, 9), 2.0), ("library ramp", (1, 11), 3.0), ("lab", (5, 11), 1.0),
         ("clinic ramp", (2, 7), 3.0), ("gym", (6, 2), 1.0)]

import heapq
from itertools import permutations

def astar(a, b):
    """A* on the grid with Manhattan distance; returns (path length in cells, nodes expanded)."""
    h = lambda p: abs(p[0] - b[0]) + abs(p[1] - b[1])
    frontier, g, expanded = [(h(a), 0, a)], {a: 0}, 0
    while frontier:
        _, gc, p = heapq.heappop(frontier)
        if p == b: return gc, expanded
        if gc > g[p]: continue
        expanded += 1
        for dr, dc in ((1, 0), (-1, 0), (0, 1), (0, -1)):
            q = (p[0] + dr, p[1] + dc)
            if q in FREE and gc + 1 < g.get(q, 1e9):
                g[q] = gc + 1; heapq.heappush(frontier, (gc + 1 + h(q), gc + 1, q))

points = [("depot", DEPOT, 0)] + TASKS
dist, total_expanded = {}, 0
for (n1, p1, _), (n2, p2, _) in permutations(points, 2):
    dist[(n1, n2)], e = astar(p1, p2); total_expanded += e
print(f"A* computed {len(dist)} shortest paths, expanding {total_expanded} nodes in total")

def score(order):
    """Travel time plus priority-weighted waiting time: an entrance cleared later costs more."""
    t, waiting, here = 0, 0.0, "depot"
    for name in order:
        t += dist[(here, name)]; here = name
        waiting += dict((n, w) for n, _, w in TASKS)[name] * t
    return t, waiting

names = [n for n, _, _ in TASKS]
best_travel = min(permutations(names), key=lambda o: score(o)[0])
best_weighted = min(permutations(names), key=lambda o: score(o)[1])
for label, order in (("shortest tour", best_travel), ("priority-weighted", best_weighted)):
    t, w = score(order)
    print(f"{label:18} {' > '.join(order)}\n{'':18} travel {t} cells, weighted waiting {w:.0f}")

A* computed 30 shortest paths, expanding 510 nodes in total
shortest tour      gym > clinic ramp > residence > lab > library ramp
                   travel 28 cells, weighted waiting 209
priority-weighted  clinic ramp > residence > library ramp > lab > gym
                   travel 36 cells, weighted waiting 165

The two objectives give different plans. The shortest tour starts with the nearby gym, a low-priority entrance, and saves eight cells of driving. The priority-weighted plan goes first to the clinic ramp, then the residence and the library ramp, and leaves the gym for last; it drives farther, but the people who most need a cleared entrance wait much less. Neither plan is wrong. They embody different value judgments, and the weights that encode those judgments (3 for an accessible ramp, 1 for the gym) were chosen by people, not discovered by the algorithm. We return to this point in the ethics section.

Part 2: belief and the decision to salt

At each entrance Plowbot must decide whether to salt. Before reading its sensor it believes, from the forecast, that a given entrance is icy with probability 0.35. Each sensor reading updates that belief by Bayes' rule from lesson 9. The decision then follows lesson 10: salting costs 10 units; skipping costs an expected 60p units, where p is the probability of ice and 60 is the expected harm of an untreated icy entrance of this priority. So Plowbot should salt exactly when 60p > 10, that is when p exceeds 1/6. Is it worth taking another reading before deciding? That is a value-of-information question.

# Belief about ice on one sidewalk segment, updated from a noisy surface sensor, and the decision to salt.
P_ICE_PRIOR = 0.35
SENSE = {True: 0.85, False: 0.10}          # P(sensor says "ice" | ice), P(sensor says "ice" | no ice)
COST_SALT, COST_FALL = 10.0, 60.0          # salting a segment vs expected harm of an untreated icy segment

def update(p, reading):
    like_ice = SENSE[True] if reading else 1 - SENSE[True]
    like_dry = SENSE[False] if reading else 1 - SENSE[False]
    return like_ice * p / (like_ice * p + like_dry * (1 - p))

def decide(p):
    eu_salt, eu_skip = -COST_SALT, -COST_FALL * p
    return ("salt" if eu_salt > eu_skip else "skip"), eu_salt, eu_skip

threshold = COST_SALT / COST_FALL
print(f"salt whenever P(ice) > {threshold:.3f}")
p = P_ICE_PRIOR
print(f"prior P(ice) = {p:.3f} -> {decide(p)[0]}")
for reading in (True, False, False):
    p = update(p, reading)
    print(f"sensor {'ice  ' if reading else 'clear'} -> P(ice) = {p:.3f} -> {decide(p)[0]}")

# Is one more sensor pass worth its cost? Value of the information from a single reading.
def value_with_reading(p):
    p_says_ice = SENSE[True] * p + SENSE[False] * (1 - p)
    return sum(pr * max(decide(update(p, r))[1:]) for r, pr in ((True, p_says_ice), (False, 1 - p_says_ice)))
for p in (0.05, 0.17, 0.35, 0.80):
    print(f"P(ice) = {p:.2f}: value of one more reading = {value_with_reading(p) - max(decide(p)[1:]):.2f}")

salt whenever P(ice) > 0.167
prior P(ice) = 0.350 -> salt
sensor ice   -> P(ice) = 0.821 -> salt
sensor clear -> P(ice) = 0.433 -> salt
sensor clear -> P(ice) = 0.113 -> skip
P(ice) = 0.05: value of one more reading = 1.17
P(ice) = 0.17: value of one more reading = 6.19
P(ice) = 0.35: value of one more reading = 3.22
P(ice) = 0.80: value of one more reading = 0.00

In the sequence shown, an 'ice' reading pushes the belief from 0.35 to 0.82, and the two 'clear' readings that follow bring it down to 0.43 and then 0.11, below the threshold, so Plowbot finally skips salting. The value of one more reading is highest, about 6.2 units, when the belief is near the decision threshold, where a reading is most likely to change the action. It falls when the belief is far from the threshold, and it is exactly zero at p = 0.8, where no possible reading would change the decision to salt. An agent that takes extra readings only when their value exceeds their cost (in time and battery) spends its sensing effort where it matters, which is the myopic information-gathering strategy of lesson 10.

Part 3: the whole agent

The full agent plans its route once at the depot, then at each entrance takes one sensor reading, updates its belief and maximizes expected utility, with the harm of untreated ice scaled by the entrance's priority. The baseline is a fixed routine typical of a simple reflex system: visit the entrances in a fixed list order and salt every one. Both run on the same 40 seeded storm nights, in each of which every entrance is independently icy with probability 0.35.

# The capstone domain: a sidewalk-clearing robot on an invented campus after an overnight storm.
CAMPUS = [
    "D....#......",
    ".##..#.###..",
    ".##.......#.",
    "....###.#.#.",
    ".#....#.#...",
    ".#.##...###.",
    "...#..#.....",
]  # D = depot, # = building, . = path (each cell is a 10 m sidewalk segment)
H, W = len(CAMPUS), len(CAMPUS[0])
FREE = {(r, c) for r in range(H) for c in range(W) if CAMPUS[r][c] != "#"}
DEPOT = (0, 0)
# Entrances that must be cleared: name, cell, priority weight (accessible entrances weigh more)
TASKS = [("residence", (4, 9), 2.0), ("library ramp", (1, 11), 3.0), ("lab", (5, 11), 1.0),
         ("clinic ramp", (2, 7), 3.0), ("gym", (6, 2), 1.0)]

# The whole agent on 40 seeded storm nights, against a fixed-routine baseline.
import random, heapq
from itertools import permutations

def astar_len(a, b):
    h = lambda p: abs(p[0] - b[0]) + abs(p[1] - b[1])
    frontier, g = [(h(a), 0, a)], {a: 0}
    while frontier:
        _, gc, p = heapq.heappop(frontier)
        if p == b: return gc
        if gc > g[p]: continue
        for dr, dc in ((1, 0), (-1, 0), (0, 1), (0, -1)):
            q = (p[0] + dr, p[1] + dc)
            if q in FREE and gc + 1 < g.get(q, 1e9):
                g[q] = gc + 1; heapq.heappush(frontier, (gc + 1 + h(q), gc + 1, q))

SENSE = {True: 0.85, False: 0.10}
COST_SALT, COST_FALL = 10.0, 60.0
names = [n for n, _, _ in TASKS]; cell = {n: p for n, p, _ in TASKS}; weight = {n: w for n, _, w in TASKS}
D = {(a, b): astar_len(pa, pb) for a, pa in [("depot", DEPOT)] + list(cell.items())
                               for b, pb in [("depot", DEPOT)] + list(cell.items())}

def weighted_wait(order):
    t, here, total = 0, "depot", 0.0
    for n in order:
        t += D[(here, n)]; here = n; total += weight[n] * t
    return total
PLANNED = min(permutations(names), key=weighted_wait)          # the route plan, computed once

def night(rng, smart):
    icy = {n: rng.random() < 0.35 for n in names}             # hidden state, different every night
    order = PLANNED if smart else names                        # the baseline visits entrances in list order
    salt = harm = 0.0; t = 0; here = "depot"; waiting = 0.0
    for n in order:
        t += D[(here, n)]; here = n; waiting += weight[n] * t
        if smart:                                              # sense, update belief, maximize expected utility
            reading = rng.random() < SENSE[icy[n]]
            like_ice = SENSE[True] if reading else 1 - SENSE[True]
            like_dry = SENSE[False] if reading else 1 - SENSE[False]
            p = like_ice * 0.35 / (like_ice * 0.35 + like_dry * 0.65)
            do_salt = COST_SALT < COST_FALL * weight[n] / 2 * p
        else:
            do_salt = True                                     # baseline: salt everything
        if do_salt: salt += COST_SALT
        elif icy[n]: harm += COST_FALL * weight[n] / 2         # untreated ice at a busier entrance does more harm
    return salt, harm, waiting

for smart in (False, True):
    rng = random.Random(366)
    runs = [night(rng, smart) for _ in range(40)]
    s, h, w = (sum(r[i] for r in runs) / 40 for i in range(3))
    print(f"{'rational agent' if smart else 'fixed routine '}: salt cost {s:5.1f}  expected harm {h:5.1f}  "
          f"total {s + h:5.1f}  priority-weighted waiting {w:6.1f}")

fixed routine : salt cost  50.0  expected harm   0.0  total  50.0  priority-weighted waiting  227.0
rational agent: salt cost  18.2  expected harm  14.2  total  32.5  priority-weighted waiting  165.0

The fixed routine never leaves ice untreated, but it spends 50 units of salt every night, and its arbitrary order makes high-priority entrances wait longer. The rational agent uses about 18 units of salt and accepts an expected harm of about 14 units from the occasional icy entrance its sensor misses, for a total expected cost about 35 per cent lower; its route also cuts priority-weighted waiting by more than a quarter. Whether that trade is acceptable is not a technical question. If the harm figure of 60 understates the cost of a fall on an icy ramp, the agent will under-salt; with a higher figure it salts more. The model makes the trade-off explicit and adjustable, which is a strength, but it cannot decide the numbers for us.

Engineering the agent

A deployed Plowbot would need more. It must monitor execution: if a path turns out to be blocked by a parked truck, the plan is invalid, and the agent must replan from its current position, which A* makes cheap. It should update its beliefs about neighbouring entrances from what it finds, since ice is spatially correlated, which a Bayesian network over entrances would capture. Weather changes during the night, so its decisions should be revisited, perhaps by treating the problem as an MDP or by planning repeatedly over a rolling horizon. And it must be tested: unit tests for each component, simulation over many seeded scenarios like the one above, adversarial tests with sensor failures, and staged field trials. The outputs printed in this course are reproducible because every random choice is seeded, which is also good engineering practice for any AI experiment.

Ethics: values inside the algorithms

The capstone shows in miniature why AI ethics is not an optional add-on. Every component carries value judgments. The priority weights decide who waits for a cleared entrance. The harm figure decides how much risk of injury is acceptable to save salt. The performance measure decides what counts as success at all; a robot scored only on salt used would salt nothing. These choices were made explicitly here, which is the best case. In many deployed systems they are implicit in training data, proxy metrics or default settings, and nobody notices them until harm occurs.

Larger systems raise these concerns at larger scale. Risk-assessment tools in criminal justice and lending have shown disparate error rates across groups; there are mathematical results showing that several intuitive fairness criteria cannot all be satisfied at once when base rates differ, so choosing among them is a value judgment. Facial recognition has raised privacy and accuracy concerns serious enough that several jurisdictions restricted it. Generative models raise questions of misinformation, copyright and the labour of the people whose work trained them, and the energy use of large-scale training and inference has become an environmental question. Autonomous weapons raise questions about meaningful human control over the use of force. And the field takes seriously the possibility that increasingly capable and autonomous systems could pursue misspecified objectives in ways that are hard to correct, which motivates research on alignment, interpretability and evaluation of advanced systems.

Practical frameworks translate principles into practice. Canada's Treasury Board Directive on Automated Decision-Making, in force since 2019, requires federal institutions to complete an Algorithmic Impact Assessment before deploying a system that makes or supports administrative decisions, with stronger requirements (peer review, human involvement, explanations) at higher impact levels. In the United States, the National Institute of Standards and Technology published the voluntary AI Risk Management Framework in 2023, organized around the functions govern, map, measure and manage. Professional codes, such as the ACM Code of Ethics, also apply to AI practitioners.

Governance in Canada and the United States

Canada was the first country to adopt a national AI strategy, the Pan-Canadian Artificial Intelligence Strategy, launched in 2017 and led by CIFAR. It funded three national AI institutes, Amii in Edmonton, Mila in Montréal and the Vector Institute in Toronto, and the Canada CIFAR AI Chairs program that recruits and retains researchers. On regulation, the proposed Artificial Intelligence and Data Act (AIDA), part of Bill C-27 introduced in 2022, would have regulated high-impact AI systems; it died on the order paper when Parliament was prorogued in January 2025. In 2025 the federal government appointed its first Minister of Artificial Intelligence and Digital Innovation, and in June 2026 it released a new national AI strategy, 'AI for All', a five-year plan emphasizing trust, AI literacy, sovereign computing capacity and commercialization, with privacy and online-safety measures rather than a general AI statute. The Canadian AI Safety Institute, created in 2024, studies the risks of advanced systems.

South of the border, United States federal policy shifted in 2025 toward removing barriers to AI development, and a December 2025 executive order sought to limit state AI laws in favour of a single national framework. The European Union took the most comprehensive regulatory path with its AI Act, which entered into force in 2024 and phases in obligations for high-risk systems over several years. These approaches change quickly; anyone deploying AI should check the current rules in each jurisdiction rather than rely on any course, including this one.

The field today

AI in the mid-2020s is dominated by large neural networks trained on vast data: language models that write and converse, image and video generators, speech systems, and increasingly agents that use tools and act on behalf of users. Recognition has followed. In 2024 the Nobel Prize in Physics went to John Hopfield and Geoffrey Hinton for foundational work on neural networks, and the Nobel Prize in Chemistry was shared by David Baker with Demis Hassabis and John Jumper, whose AlphaFold predicts protein structures. The 2024 Turing Award, announced in 2025, recognized Andrew Barto and Richard Sutton for reinforcement learning.

The classical methods of this course are not displaced by these developments; they are often inside them or beside them. AlphaZero is Monte Carlo tree search with a learned evaluator. Reasoning in language models is increasingly framed as search over possible chains of thought, sometimes guided by learned value estimates, an echo of lesson 3. Tool-using agents call planners, constraint solvers, SAT solvers and databases when guarantees matter, and verify outputs against formal specifications. Probabilistic reasoning and decision theory are how such systems should handle uncertainty and act on it. Knowledge graphs and ontologies ground facts. The most effective systems combine learning, which excels at perception and pattern recognition, with search and reasoning, which excel at guarantees, combinatorics and explicit trade-offs.

Where to go next depends on interests. CMPUT-367 and CMPUT-466 cover machine learning and deep learning. AI-320 extends probabilistic models, AI-330 reinforcement learning, and AI-470 agents built around large models. ROB-310 and ROB-340 apply Bayes filters and planning to robots, and CMPUT-325 builds the logic programming machinery behind lesson 8. Whatever the path, the habits of this course carry over: formulate the problem precisely, choose an algorithm whose assumptions match the problem, measure its behaviour, and state the values encoded in its objective.

Worked examples