Capstone: responsible deep learning and an end-to-end project

This final lesson draws the course together. It first carries a complete deep learning project through every stage, with each technique from the earlier lessons in its place: careful data splits, normalization fitted on training data, He initialization, dropout, AdamW with warmup, early stopping on validation loss, a single test evaluation, calibration, per-class analysis and a robustness check. It then turns to the responsibilities that come with building systems this capable: their energy and carbon costs, the provenance of their training data and the unsettled law of copyright, fairness and misuse, and the Canadian and US governance landscape as of 2026. It ends with a map of where to go next.

The project: plan before you train

A deep learning project begins long before any training. The plan should state the task and how success will be measured, including the metric that matters to the people affected, not just accuracy; describe the data, how they were collected and labelled, and what they may be missing; set a baseline to beat; and fix the evaluation protocol in advance, with a test set that will be touched once. For this capstone the task is classifying the 1,797 8 × 8 handwritten digit images bundled with scikit-learn (originally from the UCI repository, written by 43 people). They are small enough to train on in seconds with NumPy, which keeps every step visible. A CMPUT-367-style baseline, logistic regression on standardized pixels, reaches about 96 to 97 per cent accuracy on held-out data, so a network must do better than that to justify its complexity.

The split is fixed first: 360 images for testing, 360 for validation and the remaining 1,077 for training, chosen by a seeded random permutation. Standardization statistics are computed on the training images only and then applied to all three sets, the same leakage discipline as in CMPUT-367. With more than one image per writer, a careful project would split by writer, so that no writer's style appears in both training and test data; the bundled data do not record writers, a limitation worth stating in any report.

The model and training recipe

The network is a multilayer perceptron with two hidden layers of 128 and 64 ReLU units, He-initialized (lesson 4), with dropout of 0.2 after each hidden layer (lesson 5) and a softmax output trained with stable cross-entropy (lesson 2). It is trained with AdamW, learning rate \(10^{-3}\) after a 100-step linear warmup, and decoupled weight decay of 0.01 on the weights but not the biases (lesson 3), in mini-batches of 64. After every epoch the validation loss is computed; the best parameters are kept, and training stops after 15 epochs without improvement (lesson 5). Only then is the test set used, once. A convolutional network would suit images better (lesson 6), but at 8 × 8 pixels the difference is small, and the MLP keeps the code short.

# Capstone: an end-to-end deep learning project on scikit-learn's bundled 8x8 handwritten digits, from scratch.
import numpy as np
from sklearn.datasets import load_digits
rng = np.random.default_rng(466)
X, y = load_digits(return_X_y=True); X = X / 16.0                  # pixel values scaled to [0, 1]
idx = rng.permutation(len(X)); te, va, tr = idx[:360], idx[360:720], idx[720:]   # test, validation, train
mu = X[tr].mean(0); sd = X[tr].std(0) + 1e-6                       # statistics from training data only
Z = (X - mu) / sd
sizes = [64, 128, 64, 10]
W = [rng.normal(0, np.sqrt(2 / a), (a, b)) for a, b in zip(sizes, sizes[1:])]
b = [np.zeros(n) for n in sizes[1:]]
m = [np.zeros_like(p) for p in W + b]; v = [np.zeros_like(p) for p in W + b]
def forward(Xb, train):
    hs, masks = [Xb], []
    for i in range(len(W) - 1):
        h = np.maximum(0, hs[-1] @ W[i] + b[i])
        mask = (rng.uniform(size=h.shape) >= 0.2) / 0.8 if train else np.ones_like(h)   # dropout 0.2
        hs.append(h * mask); masks.append(mask)
    s = hs[-1] @ W[-1] + b[-1]; s -= s.max(1, keepdims=True)
    return hs, masks, s - np.log(np.exp(s).sum(1, keepdims=True))
def evaluate(rows):
    _, _, logp = forward(Z[rows], False)
    return -logp[np.arange(len(rows)), y[rows]].mean(), np.mean(logp.argmax(1) == y[rows]), logp
best, patience, t = (np.inf, None, 0), 0, 0
for epoch in range(1, 201):
    order = rng.permutation(tr)                                   # reshuffle every epoch
    for start in range(0, len(tr), 64):                           # mini-batches of 64
        rows = order[start:start + 64]
        hs, masks, logp = forward(Z[rows], True)
        g = np.exp(logp); g[np.arange(len(rows)), y[rows]] -= 1; g /= len(rows)
        grads_W, grads_b = [None] * len(W), [None] * len(W)
        for i in reversed(range(len(W))):                         # backward pass
            grads_W[i] = hs[i].T @ g; grads_b[i] = g.sum(0)
            if i > 0:
                g = (g @ W[i].T) * masks[i - 1] * (hs[i] > 0)
        t += 1; lr = 1e-3 * min(1, t / 100)                       # AdamW with a short warmup
        for j, (p, gp) in enumerate(zip(W + b, grads_W + grads_b)):
            m[j] = 0.9 * m[j] + 0.1 * gp; v[j] = 0.999 * v[j] + 0.001 * gp ** 2
            p -= lr * (m[j] / (1 - 0.9 ** t)) / (np.sqrt(v[j] / (1 - 0.999 ** t)) + 1e-8)
            if j < len(W): p -= lr * 0.01 * p                     # decoupled weight decay on weights only
    vloss, vacc, _ = evaluate(va)
    if vloss < best[0] - 1e-4:
        best, patience = (vloss, ([w.copy() for w in W], [c.copy() for c in b]), epoch), 0
    else:
        patience += 1
    if patience == 15:
        break
W, b = best[1]
print(f"stopped after epoch {epoch}; best validation loss {best[0]:.4f} at epoch {best[2]}")
loss, acc, logp = evaluate(te)
p = np.exp(logp); conf = p.max(1); correct = p.argmax(1) == y[te]
bins = np.linspace(0, 1, 11); ece = 0.0
for lo, hi in zip(bins[:-1], bins[1:]):                           # expected calibration error
    sel = (conf > lo) & (conf <= hi)
    if sel.any(): ece += sel.mean() * abs(correct[sel].mean() - conf[sel].mean())
print(f"test set (evaluated once): accuracy {acc:.3f}, loss {loss:.3f}, expected calibration error {ece:.3f}")
per_class = [np.mean(correct[y[te] == k]) for k in range(10)]
print("per-digit test accuracy:", " ".join(f"{k}:{a:.2f}" for k, a in enumerate(per_class)))
shift = Z[te] + rng.normal(0, 1.0, Z[te].shape)                   # a simple corruption: heavy pixel noise
_, _, lp = forward(shift, False)
print(f"accuracy on the same test images with added noise: {np.mean(lp.argmax(1) == y[te]):.3f}")

stopped after epoch 101; best validation loss 0.0286 at epoch 86
test set (evaluated once): accuracy 0.989, loss 0.054, expected calibration error 0.010
per-digit test accuracy: 0:1.00 1:1.00 2:0.97 3:1.00 4:0.97 5:1.00 6:1.00 7:1.00 8:0.97 9:0.97
accuracy on the same test images with added noise: 0.856

Reading the results

The network beats the linear baseline on the test set, and early stopping halted training well before the 200-epoch budget. But the single accuracy figure is the beginning of the evaluation, not the end. The expected calibration error, the average gap between the model's confidence and its actual accuracy across confidence bins, is small, so its probabilities can be read roughly at face value; large networks trained long are often overconfident, and temperature scaling (Chuan Guo and colleagues, 2017), which divides the logits by a constant fitted on validation data, is the standard fix. The per-digit accuracies show where the errors fall; here they are few and spread over several digits, and with only about 36 test images per digit, a difference of one image moves a digit's accuracy by almost 3 points, so such comparisons need more data or repeated splits before they mean much. And the noise test shows how quickly performance degrades when inputs change: accuracy drops substantially when strong pixel noise is added, a reminder from lesson 11 that held-out accuracy on clean data says little about robustness. A report on this model would state all of these, together with the data's limitations.

In a real project, the next steps would follow from these results: try a small convolutional network and data augmentation (small shifts and rotations, not flips, for digits), train with noise augmentation if noisy inputs are expected in deployment, run several seeds to put error bars on the comparison with the baseline, and document the model with a model card describing its intended use, training data, evaluation results by class and condition, and known limitations (Margaret Mitchell and colleagues, 2019).

The costs of computation

Training large models uses a great deal of energy. Emma Strubell, Ananya Ganesh and Andrew McCallum (2019) drew attention to the carbon footprint of NLP research, and later work by David Patterson and colleagues at Google (2021) showed that the emissions of a training run vary by orders of magnitude with the efficiency of the hardware, the data centre's overhead (its power usage effectiveness, or PUE) and above all the carbon intensity of the electricity grid. A run on a hydro-dominated grid can emit a small fraction of the same run on a coal-heavy grid. Inference matters too: a model served to millions of users can consume more energy over its lifetime than its training did. Reporting energy use and emissions alongside results, choosing efficient architectures and smaller models where they suffice, and using tools such as parameter-efficient fine-tuning instead of training from scratch are practical steps every practitioner can take. Data-centre electricity demand from AI was a significant energy-policy question in both Canada and the United States in 2025 and 2026, with debates over grid capacity, new generation and siting.

# Back-of-envelope energy and emissions for a training run (all inputs are assumptions to replace with your own).
gpus, hours, watts, pue = 64, 72, 700, 1.2          # accelerators, wall-clock hours, board power, facility overhead
kwh = gpus * hours * watts / 1000 * pue
print(f"energy: {gpus} accelerators x {hours} h x {watts} W x PUE {pue} = {kwh:,.0f} kWh")
for grid, g_per_kwh in (("hydro-dominated grid (illustrative)", 30), ("mixed grid (illustrative)", 400), ("coal-heavy grid (illustrative)", 750)):
    print(f"  {grid:36}: about {kwh * g_per_kwh / 1e6:6.1f} t CO2e")
flops = 6 * 1.3e9 * 26e9                            # the 6ND rule for a 1.3 B model on 26 B tokens
peak, util = 989e12, 0.4                            # dense 16-bit peak per accelerator (illustrative) and achieved utilization
print(f"compute check: {flops:.2e} FLOPs needs about {flops / (peak * util) / 3600:,.0f} accelerator-hours at {util:.0%} utilization")

energy: 64 accelerators x 72 h x 700 W x PUE 1.2 = 3,871 kWh
  hydro-dominated grid (illustrative) : about    0.1 t CO2e
  mixed grid (illustrative)           : about    1.5 t CO2e
  coal-heavy grid (illustrative)      : about    2.9 t CO2e
compute check: 2.03e+20 FLOPs needs about 142 accelerator-hours at 40% utilization

Data provenance and copyright

Large models are trained on data scraped from the web and other sources at enormous scale, and where that data came from raises legal and ethical questions: copyright, consent, privacy and the representation of the people in it. Data documentation practices such as datasheets for datasets (Timnit Gebru and colleagues, 2021) record how a data set was collected, what it contains and what it should not be used for. Audits of large web-scraped data sets have found personal information, copyrighted works, toxic content and abuse material, which has led to data sets being withdrawn and filtering pipelines being improved.

The copyright status of training on copyrighted works remained unsettled as of October 2026, with outcomes differing by case and by country. In the United States, a federal judge in Bartz v. Anthropic found in June 2025 that training on lawfully acquired books was a fair use but allowed claims over pirated copies to proceed, and the case settled in September 2025 for 1.5 billion US dollars without setting a precedent on fair use; in Kadrey v. Meta, also in June 2025, a different judge ruled for Meta on the record before him while signalling that market harm could weigh against fair use in other cases. In February 2025, in Thomson Reuters v. Ross Intelligence, a court rejected a fair use defence for a non-generative legal research tool. The New York Times's suit against OpenAI and Microsoft, filed in December 2023, was still pending in early 2026. In the United Kingdom, the High Court ruled largely for Stability AI against Getty Images in November 2025, finding that the trained model was not itself an infringing copy. In Canada, a coalition of news publishers sued OpenAI in Ontario in November 2024, and in November 2025 the Ontario Superior Court of Justice allowed the action to proceed in Ontario against OpenAI's main entities, without deciding the merits. Licensing agreements between AI developers and publishers have multiplied alongside the litigation. Anyone building on scraped data should follow these developments and take advice.

Fairness, misuse and safety

Deep networks inherit the patterns of their training data, including its biases. Joy Buolamwini and Timnit Gebru's Gender Shades study (2018) found commercial face analysis systems far less accurate for darker-skinned women than for lighter-skinned men, and language models reproduce stereotypes present in web text. The evaluation habits of this course apply directly: measure performance by subgroup, look for proxies and shortcuts, and involve the people affected in deciding which errors matter. AI-460 develops fairness measures and mitigation in depth.

Generative models add misuse concerns: synthetic images, audio and video used for fraud, harassment or political deception; automated generation of spam and disinformation; and assistance with dangerous technical knowledge. Mitigations include content provenance standards such as C2PA content credentials, watermarking of generated content, usage policies and monitoring, and safety training and evaluation of models before release. Many jurisdictions moved in 2024–2026 to criminalize non-consensual sexual deepfakes. AI-430 covers the technical side of AI safety and AI-440 the law and policy.

Governance in Canada and the United States

In Canada, the proposed Artificial Intelligence and Data Act (AIDA), part of Bill C-27, would have regulated high-impact AI systems, but it died on the order paper when Parliament was prorogued in January 2025. In June 2026 the federal government released a national AI strategy, 'AI for All', that emphasizes trust, AI literacy and commercialization rather than a general AI statute. On June 15, 2026 it introduced Bill C-36, which would replace the privacy provisions of PIPEDA with a Protecting Privacy and Consumer Data Act, including duties to explain the use of automated decision systems for decisions with significant effects; as of October 2026 the bill had not been passed. Quebec's Law 25 already requires organizations to inform people when a decision about them is made exclusively by automated processing. Canada is also home to much of the history in this course: the deep learning research of Geoffrey Hinton at the University of Toronto, Yoshua Bengio at Université de Montréal and Richard Sutton at the University of Alberta, and the national AI institutes Vector, Mila and Amii. Check the current state of these rules before relying on this summary.

Elsewhere, the European Union's AI Act, in force since August 2024, phases in obligations by risk level, including duties for providers of general-purpose AI models that began in August 2025. Rules will keep changing faster than any course can track. What does not change is the professional responsibility of the engineer who builds a model to understand what it does, measure how well it does it and for whom it fails, document its limitations honestly, and raise concerns when a system should not be deployed.

Where to go next

This course has covered the core of deep learning: networks as compositions of differentiable layers; backpropagation as reverse-mode automatic differentiation; optimizers, schedules and batch sizes; initialization, normalization and residual connections; regularization and the puzzles of generalization; convolutional, recurrent and attention-based architectures; pretraining, transfer, parameter-efficient adaptation and scaling laws; the main families of generative models; and the practice of training, interpreting and stress-testing networks. In the AI program, AI-340 and AI-350 apply these ideas to language and vision, AI-410 to large language models and generative AI, AI-420 to production systems, AI-430 to safety and alignment, AI-460 to fairness and privacy, and AI-480 to hardware and distributed training; AI-330 covers reinforcement learning. The best way to consolidate the material is to reimplement a few key papers yourself, starting with a small transformer trained on a text you care about, and to carry a project of your own through every step of this capstone, including the report.

Worked examples