Deploying vision systems: robustness, privacy and a capstone project

A model that scores well on a benchmark is the beginning of a vision system, not the end. Deployed systems must run on real cameras within cost and latency budgets, keep working as conditions change, resist deliberate attacks, protect the privacy of the people they see, treat different groups fairly, and comply with laws that are developing quickly. Lesson 9 discussed the special demands of safety-critical vision in vehicles and medicine. This final lesson covers the engineering of vision systems more broadly: system architecture and capacity planning, monitoring and continuous improvement, adversarial and physical attacks, privacy-protective design, fairness, the regulatory landscape, and a capstone project that brings the course together.

System architecture

A typical vision system is a pipeline: cameras capture images or video; frames are decoded, resized and normalized; one or more models run (for example a detector, then a tracker, then a classifier on each tracked object); post-processing applies business rules; and results are stored, displayed or trigger actions, with a human review loop for uncertain or important cases. The main architectural decision is where computation happens:

• On the device (camera, phone, robot): lowest latency, works without connectivity, keeps images local for privacy, but limited compute and harder to update.
• On an edge server on site: more compute, serves many cameras, avoids streaming video off site.
• In the cloud: elastic compute and easy updates, but bandwidth costs, latency, dependence on connectivity, and privacy and data residency concerns.

Many systems are hybrid: lightweight models on the device filter or detect events, and heavier models in the cloud analyse the selected frames.

import math

# Sizing a video analytics deployment: 120 cameras at a logistics site
cams, fps_cam = 120, 15
infer_ms = {"small detector at 640 px (edge GPU)": 9.0, "medium detector at 640 px (server GPU)": 4.5, "medium detector at 1280 px (server GPU)": 16.0}
for name, ms in infer_ms.items():
    for fps in (15, 5):
        load = cams * fps * ms / 1000                    # GPU-seconds needed per second
        gpus = math.ceil(load / 0.7)                     # keep each GPU under 70% busy
        print(f"{name:40s} at {fps:2d} fps per camera: {load:5.1f} GPU-s/s -> {gpus} GPU(s)")
# Network bandwidth: streaming all cameras to the cloud versus sending only events
bitrate_mbps = 4.0                                       # 1080p H.264 stream
print(f"streaming all cameras: {cams * bitrate_mbps:.0f} Mbit/s continuously, {cams * bitrate_mbps / 8 * 3600 * 24 / 1e6:.1f} TB per day")
events_per_hour, clip_mb = 300, 2.5
print(f"edge processing, uploading {events_per_hour} event clips per hour: {events_per_hour * clip_mb * 8 / 3600:.1f} Mbit/s average")

small detector at 640 px (edge GPU)      at 15 fps per camera:  16.2 GPU-s/s -> 24 GPU(s)
small detector at 640 px (edge GPU)      at  5 fps per camera:   5.4 GPU-s/s -> 8 GPU(s)
medium detector at 640 px (server GPU)   at 15 fps per camera:   8.1 GPU-s/s -> 12 GPU(s)
medium detector at 640 px (server GPU)   at  5 fps per camera:   2.7 GPU-s/s -> 4 GPU(s)
medium detector at 1280 px (server GPU)  at 15 fps per camera:  28.8 GPU-s/s -> 42 GPU(s)
medium detector at 1280 px (server GPU)  at  5 fps per camera:   9.6 GPU-s/s -> 14 GPU(s)
streaming all cameras: 480 Mbit/s continuously, 5.2 TB per day
edge processing, uploading 300 event clips per hour: 1.7 Mbit/s average

Capacity is driven by cameras × frame rate × inference time. Analysing 5 frames per second instead of 15, sufficient for most activity monitoring, cuts the hardware by two thirds; doubling the input resolution to catch small objects roughly quadruples it. Batching frames from several cameras raises GPU throughput above these simple estimates, which assume one frame at a time. Streaming every camera to the cloud would need nearly 500 Mbit/s continuously and over 5 TB of data a day, while processing at the edge and uploading only event clips needs under 2 Mbit/s. These trade-offs often decide the architecture before any model is chosen.

Monitoring and continuous improvement

Vision models degrade silently as the world changes: cameras are moved, replaced or get dirty, seasons and lighting change, new product designs or vehicle types appear, and user behaviour shifts (Lesson 4 discussed distribution shift). Production systems therefore monitor:

• Input health: frame rate, brightness, blur and occlusion metrics per camera, to catch failed, obstructed or moved cameras.
• Input drift: statistics of image features or embeddings compared with the training data.
• Output statistics: detection counts, class proportions and confidence distributions, which shift when something changes.
• Ground-truth samples: a small, regular sample of frames labelled by people, giving real accuracy over time.

Errors found in production, especially confident mistakes and cases flagged by users, are added to the training and test sets in a data flywheel: retrain, evaluate against the previous model on a fixed test set and on recent data, and deploy only if the new model is better, with the ability to roll back. Active learning chooses which unlabelled frames to label next, for example those where the model is uncertain or where models disagree, so labelling effort goes where it improves the model most.

Adversarial and physical attacks

Vision models can be fooled by inputs crafted to cause errors. In a digital adversarial example, every pixel is changed by a tiny, imperceptible amount in the direction that most increases the loss. The fast gradient sign method computes this direction from the gradient of the loss with respect to the input:

import numpy as np

# Adversarial examples on a linear classifier over 28 x 28 images: small, structured perturbations flip predictions
rng = np.random.default_rng(0)
d = 784
w = rng.normal(0, 1, d) / np.sqrt(d)            # weights of a trained linear classifier (illustrative)
X = rng.uniform(0, 1, (2000, d))
margin = X @ w - np.median(X @ w)                 # decision score relative to the threshold
for eps in (0.005, 0.01, 0.02, 0.05):
    # fast gradient sign method: move each pixel by eps against the current decision
    Xadv = np.clip(X - eps * np.sign(w)[None, :] * np.sign(margin)[:, None], 0, 1)
    flipped = np.mean(np.sign(Xadv @ w - np.median(X @ w)) != np.sign(margin))
    rand = np.clip(X + eps * rng.choice([-1, 1], X.shape), 0, 1)
    flipped_r = np.mean(np.sign(rand @ w - np.median(X @ w)) != np.sign(margin))
    print(f"perturbation {eps:.3f} per pixel ({eps * 255:4.1f} of 255 levels): adversarial flips {flipped:5.1%}, random noise flips {flipped_r:5.1%}")
print(f"score change from an adversarial step = eps x sum|w| = eps x {np.abs(w).sum():.1f}, versus typical margin {np.median(np.abs(margin)):.2f}")

perturbation 0.005 per pixel ( 1.3 of 255 levels): adversarial flips 30.7%, random noise flips  1.1%
perturbation 0.010 per pixel ( 2.6 of 255 levels): adversarial flips 57.9%, random noise flips  1.3%
perturbation 0.020 per pixel ( 5.1 of 255 levels): adversarial flips 88.4%, random noise flips  2.5%
perturbation 0.050 per pixel (12.8 of 255 levels): adversarial flips 100.0%, random noise flips  4.9%
score change from an adversarial step = eps x sum|w| = eps x 22.6, versus typical margin 0.19

A change of just over one intensity level per pixel, invisible to people, flips almost a third of the predictions, and five levels flip nearly 90%, while random noise of the same size barely matters. The explanation, given by Goodfellow and colleagues in 2014, is that in high dimensions many tiny changes, each aligned with the model's weights, add up to a large change in the output: the shift equals \(\epsilon\) times the sum of the absolute weights, which grows with the number of pixels. Deep networks are similarly vulnerable.

Physical attacks are more relevant to deployed systems: printed adversarial patches, stickers on stop signs, patterned clothing that hides people from detectors, and projected light. Defences include adversarial training (training on adversarial examples, which improves robustness at some cost in accuracy), input preprocessing, ensembles, certified robustness methods for small perturbations, and, most importantly, system design: multiple sensors with different failure modes, plausibility checks (a stop sign does not appear in the middle of a highway), and human oversight for consequential decisions. Other threats include data poisoning, in which manipulated training images implant backdoors triggered by a specific pattern, and model extraction through repeated queries.

Privacy-protective design

Cameras capture people who have not agreed to be analysed. Privacy law applies to images of identifiable individuals.

In Canada, PIPEDA and the substantially similar private-sector laws of Quebec, Alberta and British Columbia govern the collection, use and disclosure of personal information, including images, by businesses; public bodies follow federal and provincial public-sector privacy laws. The Office of the Privacy Commissioner and provincial commissioners have found that facial recognition requires express consent in most commercial contexts (as in the 2021 Clearview AI findings and the 2020 Cadillac Fairview mall investigation), and Quebec's Law 25 requires disclosure of biometric systems and privacy impact assessments. Canada's proposed federal Artificial Intelligence and Data Act did not become law when its bill died on the order paper in early 2025, so AI-specific obligations currently come mainly from privacy law, sector regulators and provincial rules.

Privacy-protective design reduces the personal information a system collects and keeps:

• Collect only what the purpose needs: count people or vehicles without identifying them; use low resolution or overhead views where identity is not required.
• Process on the device or edge and transmit only results or anonymized metadata.
• Blur or mask faces and licence plates automatically before storage or review.
• Retain video for short, defined periods, and control and log access.
• Tell people with clear signage, and conduct privacy impact assessments before deployment.
• Avoid sensitive inferences, such as emotion, ethnicity or health, unless clearly justified, lawful and validated; emotion recognition from faces in particular has weak scientific support.

Fairness

Vision systems can perform worse for some groups of people. Landmark studies found that commercial gender classification systems had error rates below 1% for lighter-skinned men but over 30% for darker-skinned women (the 2018 Gender Shades study), and the US National Institute of Standards and Technology found demographic differences in false match rates across many face recognition algorithms, though the best algorithms have reduced them. Causes include unrepresentative training data, cameras and exposure settings tuned for lighter skin, and labels that encode bias. Pedestrian detectors, medical imaging models and skin-condition classifiers have shown similar gaps. Mitigation starts with measurement: evaluate performance separately for relevant groups and conditions (Lesson 4), collect representative data, set acceptance thresholds for the worst-performing group, and involve affected communities in decisions about whether and how to deploy.

Regulation of vision AI

Beyond privacy law, regulation of vision systems is expanding. The European Union's AI Act, in force since 2024 with obligations phasing in through 2027, prohibits some uses outright (including untargeted scraping of facial images to build recognition databases, emotion recognition in workplaces and schools, and, with narrow exceptions, real-time remote biometric identification in public spaces by police) and classifies others, such as biometric categorization and AI in safety components of products, as high risk, with requirements for data quality, documentation, human oversight and conformity assessment. It affects Canadian and US companies that offer systems in the EU. Sector regulators also apply: medical device regulators for diagnostic imaging (Health Canada, the FDA), transport regulators for vehicle systems, and workplace laws for employee monitoring, such as Ontario's requirement that larger employers have written electronic monitoring policies.

The capstone project

The course project builds a complete vision system for a realistic problem, in teams, with a written report, a demonstration and a short presentation. Suitable problems include detecting personal protective equipment at a work site, counting and classifying vehicles at an intersection, inspecting manufactured parts for defects, reading meters or labels, monitoring wildlife from camera traps, or extracting fields from a type of document. The project follows the lifecycle taught in the course:

• Problem definition: the decision the system supports, users, success criteria (for example recall of at least 95% for missing hard hats with at most one false alarm per camera-hour), constraints on latency, cost and hardware, and a privacy and fairness assessment.
• Data: collect or select images that represent deployment conditions, write labelling guidelines, label (using SAM or open-vocabulary models to pre-label, Lessons 6 and 11), and split by camera, site or time to avoid leakage (Lesson 4).
• Baselines: a classical or simple approach (Lesson 2), and a pretrained model or zero-shot vision-language model, to show what simple methods achieve.
• Model development: fine-tune a detector, segmenter or classifier (Lessons 3, 5 to 7), with appropriate augmentation, and calibrate its confidence.
• Evaluation: metrics matched to the decision, per-condition and per-group results, error analysis with examples, robustness tests (blur, lighting, new cameras), and calibration.
• Deployment design: architecture, capacity estimate, latency measured on target hardware, monitoring plan and human review process (this lesson).
• Documentation: a model card summarizing intended use, data, performance, limitations and ethical considerations.