Audits, governance and monitoring in practice

The previous lessons built a toolkit: fairness metrics and mitigations, privacy definitions and technologies, documentation and explanation. This final lesson puts the toolkit to work across an organization and over time. It covers impact assessments before development, internal and external audits, governance structures that assign responsibility, monitoring of fairness and privacy after deployment, incident response, and a capstone project that integrates the course. The emphasis is on evidence: a responsible AI program is judged by what it can show, not by what it claims.

Impact assessments

An algorithmic impact assessment is completed before a system is built or bought and updated before deployment. It describes the system's purpose and context, the people affected, the data used, foreseeable harms and their severity and likelihood, the fairness, privacy and transparency measures planned, human oversight arrangements, and residual risks with the name of the person who accepted them. Privacy impact assessments, required under laws such as the GDPR for high-risk processing, focus on personal data flows; algorithmic impact assessments extend the scope to fairness, transparency and broader harms. The value of the assessment lies in the questions it forces early, when changing the design is still cheap, and in the baseline it gives later auditors.

In the United States, federal agencies must assess and manage risks of rights-impacting and safety-impacting AI under Office of Management and Budget guidance, and states such as Colorado have enacted laws requiring deployers of high-risk AI systems to complete impact assessments and use reasonable care to avoid algorithmic discrimination.

Audits

An AI audit is an independent examination of whether a system meets defined criteria. Audits vary along several dimensions:

• Internal or external: internal audits by a separate team within the organization have full access but limited independence; external audits by third parties have more credibility, especially when results are published.
• Process or outcome: process audits check whether the organization followed its procedures (impact assessments, documentation, approvals); outcome audits test the system's actual behavior, for example error rates by group.
• Access level: black-box audits query the system as a user would; white-box audits examine the code, data and model; most outcome audits fall between, with query access and some documentation.
• Criteria: legal requirements, standards such as ISO/IEC 42001, the organization's own policies, or criteria agreed with affected communities.

Raji and colleagues proposed an internal audit framework, SMACTR, with stages of scoping, mapping, artifact collection, testing and reflection, producing documented evidence at each stage. External auditors such as journalists and researchers have exposed some of the most significant failures, including the Gender Shades study and analyses of hiring and advertising systems, often with only black-box access.

Audits in law and practice

Audit requirements are spreading. New York City's Local Law 144 requires employers using automated employment decision tools to commission an annual independent bias audit reporting selection rates and impact ratios by sex and race and ethnicity, and to publish a summary. The EU AI Act requires conformity assessment, risk management, data governance and post-market monitoring for high-risk systems, and the Digital Services Act requires very large online platforms to undergo independent audits. Effective audits require clear criteria, access to the necessary data, auditor independence, and follow-through: findings must lead to remediation and be re-checked.

Governance structures

Governance turns responsible AI from the concern of a few enthusiasts into an organizational routine. Common elements include:

• Policy: an AI policy stating principles, prohibited uses and requirements by risk level.
• Inventory: a register of all AI systems in use, including vendor systems, with owners and risk classification.
• Risk tiers: proportionate requirements, so low-risk systems need light documentation while high-risk systems need impact assessments, audits and approval.
• Review bodies: a responsible AI committee or review board with technical, legal, ethics and domain members that approves high-risk systems.
• Roles: named system owners, model developers, independent validators, and an executive accountable for the program.
• Training: teaching staff to recognize risks and use the tools.
• Procurement: requirements for vendors to provide documentation, test results and audit rights.

The NIST AI Risk Management Framework organizes these activities under four functions: govern (culture and accountability), map (context and risks), measure (analysis and tracking) and manage (prioritizing and acting on risks). ISO/IEC 42001 specifies an auditable AI management system. AI-440 covers these frameworks in detail.

Monitoring after deployment

A model that was fair and accurate at launch can drift. Populations change, upstream data pipelines change, users adapt their behavior, and feedback loops (Lesson 2) reshape the data. Monitoring should track, by group where data permit:

• Input distributions, to detect data drift.
• Outcomes such as selection rates, and once labels arrive, error rates and calibration.
• Human overrides and appeals, which reveal where people disagree with the model.
• Complaints and incidents.
• Privacy indicators, such as unusual query patterns suggesting extraction.

Alerts should have defined thresholds and an owner who must respond.

import numpy as np
# Monitoring a deployed model: monthly fairness metrics with alert thresholds
rng = np.random.default_rng(12)
months = 12
print("month  n_B   sel_ratio  TPR_A  TPR_B  TPR_gap  status")
for mth in range(1, months + 1):
    n = 4000
    b = rng.random(n) < 0.3
    y = rng.random(n) < 0.4
    drift = 0.0 if mth < 7 else 0.015 * (mth - 6)        # from month 7 a data pipeline change degrades a feature for group B
    noise = np.where(b, 0.15 + drift, 0.15)
    score = 0.5 * y + 0.25 + rng.normal(0, 1, n) * noise
    score = score - np.where(b, drift * 0.5, 0)
    pred = score > 0.5
    sel_ratio = pred[b].mean() / pred[~b].mean()
    tpr_a, tpr_b = pred[~b & y].mean(), pred[b & y].mean()
    gap = tpr_a - tpr_b
    status = "ALERT" if gap > 0.05 or sel_ratio < 0.8 else ("watch" if gap > 0.03 else "ok")
    print(f"{mth:5d}  {b.sum():4d}  {sel_ratio:9.2f}  {tpr_a:5.2f}  {tpr_b:5.2f}  {gap:7.2f}  {status}")

month  n_B   sel_ratio  TPR_A  TPR_B  TPR_gap  status
    1  1207       1.01   0.95   0.94     0.02  ok
    2  1186       1.02   0.95   0.97    -0.01  ok
    3  1234       1.02   0.95   0.97    -0.02  ok
    4  1190       1.03   0.95   0.97    -0.01  ok
    5  1273       0.98   0.95   0.94     0.01  ok
    6  1217       1.06   0.95   0.96    -0.01  ok
    7  1207       0.98   0.95   0.91     0.05  watch
    8  1224       0.98   0.95   0.87     0.08  ALERT
    9  1178       0.98   0.96   0.89     0.07  ALERT
   10  1177       0.91   0.96   0.84     0.12  ALERT
   11  1196       0.94   0.95   0.82     0.13  ALERT
   12  1207       0.88   0.95   0.77     0.17  ALERT

For six months the model performs equally well for both groups. In month 7 a change in an upstream data pipeline starts degrading a feature for group B. The true positive rate gap reaches 0.05 and the status moves to "watch"; from month 8 it exceeds the 0.05 alert threshold and grows to 0.17 by month 12. Notice that the selection rate ratio stays above 0.8 until the end: a monitor watching only demographic parity would have missed the problem for the whole year, because the model was still selecting group B at a similar rate while selecting the wrong people within it. This is why monitoring plans track several metrics and why error-rate metrics need labels collected after deployment, which itself requires planning.

Incident response

Despite best efforts, incidents happen: a model produces discriminatory outcomes, leaks personal data, or is used in an unintended way. An AI incident response plan defines how to detect and report incidents, who triages them, criteria for pausing or rolling back a system, how affected people are notified and offered remedies, how regulators are informed where required (for example data breach notification laws), and how lessons are captured. Public incident databases, such as the AI Incident Database, collect thousands of reported failures and are a useful source of scenarios for tabletop exercises.

Capstone project

The capstone integrates the course. Choose a realistic decision system, such as tenant screening, school admissions, insurance claims triage or a customer service assistant built on a language model, and produce:

• An impact assessment covering purpose, affected people, harms and oversight.
• A datasheet for the data you use, real or synthetic.
• A model and a disaggregated evaluation using at least three fairness metrics, with bootstrap confidence intervals and an intersectional analysis.
• A mitigation experiment comparing at least two techniques on a fairness and accuracy chart, with a justified recommendation.
• A privacy analysis: a membership inference test, and a differentially private version of at least one release or the model, with the accuracy cost.
• Explanations: global importance and a counterfactual explanation for a rejected case.
• A model card and a monitoring plan with metrics, thresholds and owners.

The deliverable should read like the evidence package an auditor or regulator would request: specific, quantitative, honest about limitations and clear about who decided what.

Engaging affected communities

Governance processes often involve only the organization and its regulators. Engaging affected communities adds perspectives that neither has. Practical forms include advisory panels of people affected by a system, consultation on impact assessments before deployment, participatory workshops to identify harms and choose fairness criteria, and accessible channels for feedback and complaints after launch. Engagement is most valuable when it can change decisions, when participants are compensated for their time, and when the organization reports back on what it did with the input. Public sector systems, which often affect people who cannot choose another provider, have particular reasons to engage communities, and some jurisdictions require public consultation for high-impact systems.

Measuring the program itself

A responsible AI program should measure its own effectiveness. Useful indicators include the share of AI systems in the inventory with a current impact assessment and model card, the share of high-risk systems that have passed an independent audit, the time taken to resolve monitoring alerts and incidents, the number of systems modified or withdrawn as a result of reviews, and staff completion of training. These numbers, reported regularly to senior leadership and, where appropriate, publicly in transparency reports, show whether the program is functioning or exists only on paper.

Decommissioning

Governance also covers the end of a system's life. When a model is retired, the organization should record why, notify users and affected people where appropriate, archive the documentation and evaluation records needed for future audits or legal claims, and delete or retain training data and model weights according to the retention policy. Systems that are quietly abandoned but left running, or whose data are kept indefinitely after the purpose has ended, create both privacy and accountability risks.