AI Capstone Project
A team project that takes an AI system from problem to evaluated, documented deployment: choosing and scoping a project, framing and metrics, responsible data, leakage-free splits, baselines and experiments, error analysis, LLM-based systems, rigorous evaluation, safety and fairness review, deployment and monitoring, cost and reporting, and presentation and launch evaluation.
AI-499 Year 4 3 credits 12 lessons · 4 h 59 min
Offered by HSM University · Artificial Intelligence
This is the Canadian version of this course. See the American version.
What you'll learn
- Choose, compare and scope an AI capstone project with a team charter, success criteria and a realistic plan
- Frame a user need as a machine learning task with requirements, cost-aware metrics, guardrails and targets
- Source, license, audit, label and document data responsibly, respecting privacy law and research ethics
- Design training, validation and test splits that match deployment and prevent leakage
- Build baselines, run controlled experiments and ablations, and make results reproducible
- Improve a system through end-to-end builds, error analysis, slice evaluation and data-centric fixes
- Build and evaluate systems based on large language models, including retrieval, judges and evaluation sets
- Evaluate rigorously with confidence intervals, paired comparisons, calibration and robustness tests
- Conduct a safety, fairness and risk review with red teaming, a model card and a risk register
- Deploy, test and monitor a system, detect drift and plan maintenance and handover
- Estimate operating cost and write a reproducible repository and honest technical report
- Present and demonstrate the system, plan pilots and A/B tests, and reflect on lessons learned
Course content
12 lessons · 4 h 59 min total · final exam
- 1 The capstone: choosing and scoping a project 25 min · quiz
- 2 Problem framing, requirements and metrics 25 min · quiz
- 3 Data: sourcing, permissions, quality and documentation 25 min · quiz
- 4 Splits, validation design and leakage 25 min · quiz
- 5 Baselines, experiment design and reproducibility 25 min · quiz
- 6 Building the system: iteration and error analysis 25 min · quiz
- 7 Building and evaluating systems with large language models 25 min · quiz
- 8 Rigorous evaluation: uncertainty, calibration and robustness 25 min · quiz
- 9 Safety, fairness and risk review 25 min · quiz
- 10 Deployment, serving and monitoring 25 min · quiz
- 11 Cost, documentation and the technical report 24 min · quiz
- 12 Presentation, launch evaluation and reflection 25 min · quiz
- E Final exam 20 questions
Requirements
Recommended background. You can start any course at any time.
Description
A team project that takes an AI system from problem to evaluated, documented deployment: choosing and scoping a project, framing and metrics, responsible data, leakage-free splits, baselines and experiments, error analysis, LLM-based systems, rigorous evaluation, safety and fairness review, deployment and monitoring, cost and reporting, and presentation and launch evaluation.
Part of the Science stream, typically taken in year 4.