Risk management for AI initiatives

Lorraine liked registers. Every serious risk at Cascade Ridge, from a power failure at the emergency hospital to a key supplier going out of business, was in one, with an owner, a score and a response. "So where's AI?" she asked at the start of the sixth session. It was not there. The sixth session built the AI part of the register and, with it, a way of deciding how much control each AI use needed.

What is different about AI risk

AI systems share the ordinary risks of any technology: outages, security breaches, cost overruns, vendor failure. They also bring risks of their own. Their outputs are probabilistic, so they will sometimes be wrong. Their performance can change over time without anyone touching them, as the world they model changes. They can treat groups of people unequally. Generative systems can invent facts, produce harmful content or reproduce material someone else owns. And people tend to trust a fluent, confident answer more than they should.

Domain 3 of the exam guide has a task on identifying enterprise AI risks and directing their mitigation. It asks you to recognize the need for risk controls and monitoring of AI systems in production; to understand that bias can arise at several stages of the life cycle and why monitoring for bias drift matters; to manage risks from harmful content and intellectual property concerns; and to identify and mitigate reliability risks such as hallucinations, degrading data quality and model drift. Another task asks you to apply AI risk classification frameworks to prioritize governance and compliance decisions across the life cycle.

A catalogue of AI risks

Bias needs a closer look because it can enter at many points. It can come from the data used to train a model, if past decisions were themselves unequal or if some groups are under-represented. It can come from the choice of what the model predicts: predicting who will miss an appointment is not the same as predicting who needs help to attend. It can come from how outputs are used: the same flag could trigger a friendly extra reminder or a demand for a deposit. And it can appear later, as bias drift, when the population changes and a model that was fair at launch stops being fair. That is why the exam guide stresses ongoing monitoring rather than a single check before launch.

A fairness check in practice

Before approving the no-show pilot, the steering group asked Noor to check whether the model flagged clients at rural clinics more often than clients at city clinics. Rural clients travel further, and bad winter roads cause more missed appointments, so some difference was expected. The question was whether the difference was large enough to matter, and what the flag would lead to.

The rule of thumb used in the example is only a screening device, not a legal test or a guarantee of fairness. Its value is that it prompts a question: why is there a difference, and does the use of the output harm anyone? The answer depends on the use. The same gap in flag rates would have been unacceptable if a flag meant paying a deposit.

Scoring risks

Lorraine's register used a familiar method: score each risk for likelihood and impact on a scale of 1 to 5, multiply them, and sort by the result. Scores of 15 or more were high and needed a named response and monthly review; 8 to 14 were medium, reviewed quarterly; 7 or less were low. Andre and Hannah added the AI risks.

Scores like these are judgements, and two people can disagree by a point. That is acceptable. The value of scoring is that it forces the conversation, makes priorities visible, and lets the register be reviewed as conditions change. Transcript exposure has a low likelihood only because of controls already planned, such as processing in approved services with restricted access; if those controls slipped, its score would rise.

Risk classification frameworks

Scoring individual risks is useful, but organizations also need a way to classify whole AI uses by their level of risk, so that the right amount of governance is applied to each. The exam guide asks you to apply AI risk classification frameworks to prioritize governance and compliance decisions across the life cycle. Several public frameworks help. The European Union's AI Act sorts systems into unacceptable, high, limited and minimal risk, with obligations rising by level. The NIST AI Risk Management Framework in the United States organizes risk work into four functions: govern, map, measure and manage. ISO/IEC 42001 sets out requirements for an AI management system that an organization can be certified against. None of these was written for a veterinary group, but their logic transfers.

Cascade Ridge adopted three internal tiers, based on who could be affected, how badly, and how quickly a mistake would be caught.

A tier is assigned at intake, reviewed before the pilot, and reviewed again whenever the use changes. Changing the assistant to answer clinical questions, for example, would move part of it into a more demanding category and trigger a new review. Classification across the life cycle means exactly that: the tier follows the use, not the technology, and it is checked each time the use or the context changes.

Monitoring in production

Many AI risks only show themselves after launch. The exam guide asks you to recognize the need for controls and monitoring of AI systems in production. Each initiative needed a short monitoring plan naming what would be watched, how often, by whom, and what would trigger action.

AWS provides tools that support this kind of monitoring, such as Amazon SageMaker Model Monitor for tracking data and model quality, SageMaker Clarify for bias metrics, Amazon CloudWatch for alarms, and the logging built into Amazon Bedrock. A business strategist does not need to configure them, but should know that monitoring is a planned, funded activity with an owner, not something that happens by itself.

Incidents

Some risks will occur despite the controls. An AI incident is an event in which an AI system caused or nearly caused harm, such as a wrong clinical answer, a data exposure or an unfair outcome. Cascade Ridge folded AI incidents into its existing incident process, with three additions: anyone could report an AI concern through a simple form; each AI system's owner would be told within a working day; and the steering group would review every AI incident and near miss, with the lessons fed into training and into the system's monitoring plan. The receptionist's email from lesson 6 was recorded as the first near miss.