Governing AI: U.S. law, global rules and a capstone case

On the last night of the seminar, Riverton General's board meets to decide on Pathfinder, and it has asked Varga's class for a written analysis. Raúl will present it. Over eleven sessions, the class has looked at the system through every lens the course offers: its values, its use of patient data, its fairness, its explainability, its effect on nurses, who would be responsible if it failed and whether its objective captures what the hospital wants. What remains is the question of rules: what the law requires, what it should require, and how a public institution should govern a powerful tool when the law is unsettled. Varga splits the session into two parts: first the governance landscape, then the capstone.

Why governance is hard

Governing AI is difficult for reasons the course has already met. The Collingridge dilemma means rules made early may miss the real problems, and rules made late may be too late. The pacing problem, a related idea, is that technology changes faster than law can adapt. AI is a general-purpose technology used in almost every sector, so rules designed for one use may fit another badly. Much of the expertise is concentrated in the companies to be regulated, creating a risk of regulatory capture, in which regulators come to serve the interests of the industry. And there is a genuine trade-off: rules that are too loose allow harm, and rules that are too strict or uncertain may slow beneficial uses or push development to places with weaker protections.

Governance does not mean only legislation. It includes a range of tools, from soft to hard.

• Professional and corporate self-governance: codes of ethics, internal review boards, published principles and safety policies.
• Technical standards: detailed, voluntary specifications that can later be referenced by law, such as the ISO/IEC 42001 standard for AI management systems published in 2023 and the AI Risk Management Framework published by the U.S. National Institute of Standards and Technology in the same year.
• Procurement: governments and large institutions can require suppliers to meet conditions, such as testing, documentation and audit rights, as a condition of purchase.
• Existing law applied to AI: privacy, consumer protection, human rights, product liability, negligence, employment and sector rules for areas such as health, finance and transport.
• AI-specific law: statutes that regulate AI systems as such, usually by their level of risk.
• International agreements: principles and treaties among states.

The European Union's AI Act

The most comprehensive AI-specific law is the European Union's Artificial Intelligence Act, adopted in 2024 and in force since August 2024, with its obligations phased in over several years. It takes a risk-based approach. A small set of practices is prohibited outright, including social scoring by public authorities, manipulative techniques that cause significant harm, exploiting the vulnerabilities of particular groups, untargeted scraping of facial images to build recognition databases, emotion recognition in workplaces and schools, and, with narrow exceptions, real-time remote biometric identification in public spaces for law enforcement. These prohibitions have applied since February 2025. High-risk systems, such as AI used in hiring, credit, education, essential public services, law enforcement, migration and the safety components of regulated products, must meet requirements for risk management, data quality, documentation, human oversight, accuracy and robustness. Systems that interact with people or generate content have transparency duties. Providers of general-purpose AI models have duties of documentation and copyright compliance, with additional duties for models that pose systemic risk, applying since August 2025.

In 2026 the EU amended the Act through what it called a digital omnibus, intended to simplify compliance. The amendment, in force since July 2026, postponed the main obligations for high-risk systems: for stand-alone high-risk uses, from August 2026 to December 2027, and for AI embedded in regulated products, to August 2028. It also added a new prohibition on AI systems designed to generate non-consensual intimate imagery or child sexual abuse material. Supporters said the delay gave time for standards and guidance to be ready; critics said it weakened protections under industry pressure.

Because many companies sell into the European market, EU rules often shape products used elsewhere, a pattern the legal scholar Anu Bradford has called the Brussels effect. Whether that will happen with the AI Act, as it did to a degree with the General Data Protection Regulation, is an open question.

AI governance in the United States

The United States has no comprehensive federal AI statute. In October 2023, President Biden issued Executive Order 14110, which directed federal agencies to develop standards and guidance and required developers of the most powerful models to share safety test results with the government. President Trump revoked that order on his first day in office in January 2025 and issued a new order directing agencies to remove barriers to American AI leadership. In July 2025 the administration published an AI Action Plan emphasizing innovation, infrastructure and international competitiveness. In December 2025 the President signed an executive order titled Ensuring a National Policy Framework for Artificial Intelligence, which created a Justice Department task force to challenge state AI laws considered burdensome, directed that some federal funds be conditioned on states not enforcing such laws, and called for federal legislation that would preempt conflicting state rules.

Much of the action is in the states. Colorado passed the first broad state law on high-risk AI in 2024, requiring developers and deployers of AI used in consequential decisions to take reasonable care against algorithmic discrimination. Its start was delayed, a federal court order paused its enforcement in April 2026 in a case the Justice Department supported, and in May 2026 the state replaced it with a narrower law based on notice, explanation and human review, taking effect in January 2027. California's SB 53, signed in 2025, requires large frontier model developers to publish safety frameworks and report serious incidents. New York City requires bias audits of automated hiring tools, Illinois protects biometric data, and many states regulate deepfakes. Federal agencies also apply existing law: the Federal Trade Commission has brought cases against deceptive AI claims, the Food and Drug Administration regulates AI-enabled medical devices, and civil rights and credit laws apply to automated decisions. Whether federal preemption will displace state AI laws is one of the central legal questions of the moment.

International frameworks

Several international instruments set out shared principles. The OECD AI Principles, adopted in 2019 and updated in 2024, were the first intergovernmental standard on AI and were endorsed by dozens of countries, including Canada and the United States; they call for AI that is innovative, trustworthy and respects human rights and democratic values. UNESCO's Recommendation on the Ethics of Artificial Intelligence, adopted by its member states in 2021, is the first global standard on AI ethics. The Council of Europe's Framework Convention on Artificial Intelligence and Human Rights, Democracy and the Rule of Law, opened for signature in 2024, is the first legally binding international treaty on AI; it was signed at its opening by the European Union, the United Kingdom, the United States and several other states. The G7's Hiroshima process produced a code of conduct for developers of advanced AI. And as lesson eleven described, a series of summits has addressed the safety of frontier models.

Philosophical questions about governance

Varga closes the first half of the session with questions that no law settles. Who should decide how powerful technologies are governed: elected legislatures, expert regulators, courts, companies, the people most affected, or some combination? Democratic legitimacy argues for legislatures and public participation; expertise argues for specialized regulators; the speed of change argues for flexible tools. How precautionary should governance be? A precautionary approach places the burden of proof on those who want to deploy a risky technology; an innovation-first approach places it on those who want to restrict it. Each can be defended, and the right answer may differ by domain: medicine has long required evidence of safety before use, while most software has not. And how should governance address power? Several of the course's cases, from surveillance to the distribution of productivity gains, show that the deepest questions about AI are often questions about who holds power and how it is checked.

The capstone: a full analysis of Pathfinder

Raúl presents the seminar's analysis to the board. It follows a structure the class built over the term, which can be applied to any AI system.

# The seminar's rubric for Pathfinder: each concern rated 0 (addressed) to 3 (serious, unaddressed)
concerns = {
    "benefit is real and measured against current practice": 1,
    "patient data used within the norms of care": 1,
    "errors not concentrated on any group": 2,
    "staff can see why a patient was ranked": 2,
    "nurses can override and have time to": 2,
    "clear accountable owner and liability": 1,
    "objective matches the real goal (urgency, not past admissions)": 2,
    "monitoring and power to suspend": 1,
}
total = sum(concerns.values())
print("total concern score:", total, "of", 3 * len(concerns))
for k, v in concerns.items():
    if v >= 2:
        print("must fix before use:", k)

total concern score: 12 of 24
must fix before use: errors not concentrated on any group
must fix before use: staff can see why a patient was ranked
must fix before use: nurses can override and have time to
must fix before use: objective matches the real goal (urgency, not past admissions)

• Purpose and alternatives. Pathfinder is meant to get urgent patients seen sooner. The alternatives include more triage staff, better triage protocols and using the model only as a second opinion. The test results suggest a real benefit, but they must be compared with the current process measured the same way.
• Values and stakeholders. Patients, especially those the model may under-rank; nurses, whose judgment and workload are affected; the hospital; Brightline; and the public, which funds the hospital and expects fair care.
• Privacy. The training data must stay within the norms of clinical care: used to improve emergency care, not sold or used for other products, ideally kept inside the hospital's systems, and disclosed to patients.
• Fairness. Brightline must report error rates by age, sex and, where data allow, ethnicity and other characteristics, with particular attention to under-ranking of urgent patients. The finding that the model learned from historical admissions, which may reflect unequal past treatment, is a serious concern.
• Transparency. Nurses must see the main reasons for each ranking and warnings when a patient is unlike the training data; patients must be told the model is used.
• Work and responsibility. Nurses must have the authority, time and training to override the model, must not be evaluated by how often they agree with it, and must not become a moral crumple zone. The hospital must name an accountable owner, and the contract with Brightline must allocate liability.
• Alignment. The model predicts past admission to intensive care, which is a proxy for urgency and may encode old patterns. The objective should be changed to measures closer to clinical urgency, and the model retested.
• Governance and law. The hospital should determine whether the system is a medical device subject to FDA oversight, follow HIPAA and state law, and adopt the NIST AI Risk Management Framework as its internal standard, publishing a summary of its assessment. It should adopt the system in stages, with a sunset date and a clear power to suspend.

The analysis concludes that Pathfinder should not be adopted as proposed, but could be adopted in a limited, monitored trial once four problems are fixed: the training objective, the subgroup error analysis, explanations for nurses and a protected right to override. The board votes to accept that recommendation. Theo, who started the term ready to add up outcomes, says the final recommendation is better than the one he would have made, not because it rejects the numbers but because it asks what they leave out. Sabine says it respects the patients the model gets wrong. Varga says that is what the course has been for.