AI Governance and Risk Management
Governance is what lets an AI programme grow without being stopped by its first public failure. In this module you'll learn how the EU AI Act sorts AI systems into risk tiers and what each tier demands, how to score the four categories of risk that actually derail AI programmes, how to set up a minimum viable governance structure with three essential roles and a clear escalation path, and how to test any proposed deployment against five responsible AI principles. You'll finish with a risk classification of your own AI portfolio, a risk register, a one-page governance charter and a readiness audit that shows where your governance gaps are.
You'll follow Meridian Group as its AI Programme Director, Marcus Osei, and its General Counsel, Priya Shah, put governance in place before the programme's higher-risk ideas get anywhere near deployment. This module is a leadership briefing, not legal advice: take decisions on specific systems with your own legal counsel.
AI Governance Readiness Audit
Audit your organisation's AI governance against 24 best-practice items across policy, oversight, data, risk and compliance to surface priority gaps.
5.1The EU AI Act: What Leaders Must Know
The EU AI Act (Regulation (EU) 2024/1689), in force since 1 August 2024, regulates uses of AI, not the technology. It sorts AI systems into four tiers by potential harm, and obligations follow the tier.
Prohibited practices are banned outright: social scoring, manipulative techniques that cause significant harm, emotion recognition in workplaces and schools except for medical or safety reasons, and untargeted scraping of facial images, among others. High-risk systems are allowed but tightly regulated: either safety components of products covered by EU product law (Annex I) or systems used in the Annex III areas: biometrics, critical infrastructure, education, employment and workers' management, access to essential services such as credit scoring, law enforcement, migration and border control, and the administration of justice and democratic processes. Employment covers recruitment, decisions on promotion and termination, allocating work, and monitoring and evaluating performance. An Annex III system that profiles people is always high-risk. Transparency-tier systems, often called limited risk, carry disclosure duties under Article 50: people must be told when they are dealing with a chatbot, and providers of generative AI must mark its output as AI-generated in a machine-readable way. Everything else is minimal risk.
The provider develops a system or puts it on the market under its own name; the deployer uses it. Most organisations are deployers, but if you build your own high-risk system you take on the provider's duties, including the conformity assessment. The Act also reaches beyond the EU: it applies to providers outside the EU who place AI on the EU market, and to providers and deployers anywhere whose AI output is used in the EU.
Prohibitions applied from 2 February 2025 and the rules for general-purpose AI models from 2 August 2025; Article 50's transparency duties have applied since 2 August 2026, although the Omnibus gave generative systems already on the market before that date until 2 December 2026 to add machine-readable marking. The Digital Omnibus on AI (Regulation (EU) 2026/1744, in force 27 July 2026) moved the high-risk dates: stand-alone Annex III systems now apply from 2 December 2027, and Annex I product-embedded systems from 2 August 2028. That is a delay, not a repeal. Fines for prohibited practices reach €35 million or 7% of worldwide annual turnover, whichever is higher.
The UK has no AI Act. It asks existing regulators, such as the data protection and financial services regulators, to apply five cross-sector principles from its 2023 white paper within their own remits: safety, security and robustness; transparency and explainability; fairness; accountability and governance; and contestability and redress. A UK firm whose AI output is used in the EU may have to meet the EU rules as well as UK data protection law.
5.2The Four Categories of AI Risk
Compliance is one risk among four, and rarely the one that ends an AI programme. Most programmes are damaged by an operational failure that nobody owned.
Model risk is the risk that the system is wrong: a hallucinated citation, a biased score, a forecast that drifts as the world changes. Module 1 explained why generative models produce fluent errors and predictive models drift quietly. The governance question is how you would find out. The mitigations are testing before launch, accuracy monitoring after it, and a human check wherever an error would be costly.
Data risk covers what goes into the system: poor-quality or unrepresentative data, personal data used without a lawful basis, and confidential information leaking into a vendor's model or another client's answers. AI use of personal data falls under UK GDPR or the EU GDPR, and a data protection impact assessment is usually needed when an AI system processes personal data at scale or makes decisions about people. The mitigations are documented data provenance, data minimisation, contractual limits on how vendors use your data, and access controls.
Compliance risk is regulatory and legal exposure: the AI Act, data protection, sector rules and professional duties. Intellectual property belongs here, as three questions. Was the training data lawfully used? Copyright cases against AI developers are still before the courts in the UK and the US. Who owns the output? Section 9(3) of the UK's Copyright, Designs and Patents Act 1988 gives a computer-generated work an author, the person who made the arrangements for its creation, but the government's March 2026 report on copyright and AI said it favours removing that protection for works with no human author; it has not yet legislated. The US Copyright Office's January 2025 report on copyrightability holds that purely AI-generated material isn't protected and that prompts alone aren't enough. In both countries, human creative input is what makes protection likely. Who carries the risk if output infringes? Many enterprise AI vendors now offer IP indemnities for customers' use of output, but with conditions, such as keeping the vendor's filters switched on, so read the clause first.
Reputational risk is the damage when AI behaves in ways clients, staff or the press find unacceptable, even if legal. It usually follows from one of the other three.
Bring it together in a risk register for each AI system. Score each category for likelihood and impact from 1 to 5 and multiply them, giving a score out of 25. Record the mitigations, the residual score once they are in place, and an owner. Act first on anything scoring 15 or more, and review the register whenever the system, its data or its use changes.
5.3Building Your AI Governance Structure
Governance doesn't need a bureaucracy. A mid-sized organisation needs three roles, one register and a clear escalation path.
The AI governance committee sets policy, decides on higher-risk uses and holds the programme to account. Keep it small and cross-functional: a senior executive in the chair, the AI Risk Owner, the programme lead, legal, IT and data, HR, and the heads of the main business units. It should meet monthly during an active rollout and quarterly once things settle. Give it three core mandates: approve or reject new AI uses at a depth that matches their risk; own the AI policy and the organisation's risk appetite; and report to the board, because regulators and clients will ask who at the top is accountable.
The AI Risk Owner is the one person accountable for the organisation's overall AI risk picture. They keep the AI register, run classification and risk reviews, track incidents, and can pause a deployment pending committee review. They shouldn't also lead delivery of the programme: nobody should mark their own homework.
System owners are the third role: a named business owner for each AI system, accountable for its day-to-day use: that human oversight works, performance is monitored and incidents are reported. Accountability sits with the business, not with IT or the vendor.
The AI register is a central log of every AI system in use, recording its purpose, risk tier, data, owner and next review date. You can't govern what you can't see, so include tools teams bought themselves.
The escalation path runs from system owner to AI Risk Owner to committee to board, with triggers agreed in advance: any high-risk classification, any incident involving client or personal data, a wrong output that reached a client, a sustained fall in accuracy, or an enquiry from a regulator.
Three mistakes cause governance to drift. The first is governing on paper: a policy is approved, but there is no register and nobody checks practice against it, so unofficial AI use grows around it. The second is approving once: a system is reviewed at launch and never again, although its model, data and uses change; Module 3 costed governance as a recurring expense for exactly this reason. The third is reviewing everything at the same depth. When an email drafting tool gets the same scrutiny as a hiring algorithm, the committee becomes a bottleneck, teams go around it, and the high-risk case gets no more attention than the trivial one. A light-touch route for minimal-risk and transparency-tier tools and a full review for high-risk ones keeps governance fast where it can be and strict where it must be. For a certifiable structure, see ISO/IEC 42001, the AI management system standard.
5.4Responsible AI in Practice
Principles only become useful when they turn into questions someone must answer before a system goes live. This course uses five: fairness, transparency, accountability, privacy and reliability. They condense the characteristics of trustworthy AI in the US National Institute of Standards and Technology's AI Risk Management Framework (AI RMF 1.0, January 2023), which lists seven: valid and reliable; safe; secure and resilient; accountable and transparent; explainable and interpretable; privacy-enhanced; and fair, with harmful bias managed. Other frameworks group them differently. What matters is that you choose one set and apply it to every proposal in the same way.
Fairness asks whether the system produces systematically worse outcomes for some groups. Test outcomes across groups before launch and at intervals afterwards, and ask whether the training data reflects past decisions you wouldn't want repeated. Transparency asks whether the people affected know AI is involved and whether a decision can be explained to them in terms they understand. Accountability asks whether a named person is responsible for each decision and can override the system; if nobody can say who decides, the software is deciding. Privacy asks whether personal data is minimised, lawfully used and protected, and whether a data protection impact assessment has been done where one is needed. Reliability asks whether the system has been tested against an accuracy threshold agreed for this use and is monitored after launch.
Rate each principle as met, partly met or not met. A "not met" on any system that affects people's jobs, money or rights should block deployment until it is fixed. The evaluation should produce design changes, not just a verdict. Most failing proposals can be made acceptable by changing what the AI does (rank, flag or summarise instead of decide), who sees its output, or which data it uses. Reject outright only when no design change removes the problem.
Take automated performance-review scoring: a tool that scores each employee's performance from timesheets, documents and work output. On fairness, a model built on hours recorded may penalise part-time staff and people with caring responsibilities. On transparency, employees must be told; under the EU AI Act, a deployer must inform workers and their representatives before putting a high-risk system into use in the workplace. On accountability, a score that becomes the rating fails, because no manager has decided anything. On privacy, monitoring people's work in detail is intrusive and needs a clear lawful basis. On reliability, nobody has defined what an accurate performance score would be. Two design changes help most: have the AI summarise evidence for a manager who makes and explains the rating, and remove proxy measures such as hours logged.
Priya Shah Classifies Meridian's AI Portfolio
Meridian Group's deployment plan puts five AI systems into use over 18 months, touching 340 of its 850 employees. Before the financial reporting assistant and HR onboarding chatbot go live in months 6–12, Marcus Osei and General Counsel Priya Shah classify every system against the EU AI Act. Meridian is a UK firm, but Priya uses the Act as the benchmark: it reaches any system whose output is used in the EU, and it is the strictest standard the firm faces.
Transparency tier. The HR onboarding chatbot only answers policy questions and makes no decisions about anyone's employment, so it falls outside Annex III; staff must be told they are talking to an AI. The financial reporting assistant, legal research assistant and business development email tool are generative systems whose drafts a person reviews. The research assistant stays a supervised agent, checked by a senior associate, and the email tool stays a copilot. Annex III's justice category covers AI used by or for courts, not a firm's own research.
Minimal risk. The churn model scores client organisations, not individuals, so the Act adds nothing, though UK GDPR still covers the client contacts' data it uses.
High-risk. The parked HR performance-evaluation tool would evaluate associates' performance, an Annex III employment use, and because it profiles individuals no exemption applies. Its obligations apply from 2 December 2027. If Meridian bought it, the vendor would have to complete the conformity assessment, including documentation of its training data, and Meridian, as deployer, would need trained human oversight, logs, monitoring and notice to staff before use. Building its own would make Meridian the provider. It stays parked until a conformity-assessed product has passed a responsible AI review. The later date buys time; it doesn't remove the obligation.
The process. The committee, chaired by Jane Whitfield with Priya as AI Risk Owner, sets two routes: lower-tier systems are registered by their system owner and checked by Priya; high-risk systems need full committee review, legal sign-off and a responsible AI evaluation before any pilot.
The audit. Marcus runs the readiness audit. Three of the 24 items are done (committee, Risk Owner, classification), six are partly done and fifteen not yet: (3 + 6 × 0.5) ÷ 24 = 25%, a foundational rating (below 40%). An approved AI policy, a complete register and an incident response plan head the gaps to close before months 6–12.
Knowledge Check
The foundations module showed that generative models make fluent errors while predictive models drift quietly. How should that difference shape the model-risk entries in the risk registers for Meridian's legal research assistant and its churn model?
The assistant's errors look exactly like correct answers, so its main control is a human check on every output before it reaches a client, the senior associate review; the churn model's accuracy decays as client behaviour changes, so its control is ongoing accuracy monitoring against an agreed threshold, with the register rescored when performance falls. Different failure modes need different mitigations, even under the same risk category.A board member says the Digital Omnibus on AI means high-risk rules no longer apply to the firm's planned recruitment tool. What would you tell them?
The Omnibus delayed the obligations for stand-alone Annex III systems, including employment AI, to 2 December 2027; it didn't repeal them, so the extra time should be used to build the conformity, oversight and documentation process before the tool launches.Your organisation buys a high-risk AI system from a vendor rather than building it. Which obligations stay with you?
As the deployer you must use it according to the vendor's instructions, assign trained people to oversee it, monitor it and keep its logs, and inform staff before using it in the workplace; the conformity assessment falls to the vendor as provider, unless you build your own or put your name on the system, which makes you the provider.Why does a risk register score likelihood and impact separately instead of giving each risk a single rating?
A rare but severe failure, such as a hallucinated citation reaching a client, and a frequent but trivial one need different responses; multiplying the two shows which risks deserve mitigation first, and rescoring after mitigation shows how much risk remains.A committee reviews every AI tool, from email drafting to hiring algorithms, with the same full process. What goes wrong, and what is the fix?
The committee becomes a bottleneck, teams go around it and the high-risk case gets no more attention than the trivial one; proportionate routes, a light-touch registration for minimal and transparency-tier tools and full review for high-risk ones, keep governance fast where it can be and strict where it must be.An automated performance-scoring tool fails on fairness and accountability. Why is redesigning it often better than rejecting it outright?
Most failures come from what the AI is allowed to do, not from using AI at all: changing it to summarise evidence for a manager who decides and explains the rating, and removing proxies such as hours logged, keeps the benefit while restoring human accountability and reducing bias.
Resources & Further Practice
- The Act itself: Regulation (EU) 2024/1689 (the Artificial Intelligence Act), free on EUR-Lex at eur-lex.europa.eu/eli/reg/2024/1689/oj — read Article 5 (prohibited practices), Article 6 and Annex III (high-risk), Article 26 (deployers' duties) and Article 50 (transparency).
- The amended timetable: Regulation (EU) 2026/1744 (the Digital Omnibus on AI), published in the Official Journal on 24 July 2026 and available on EUR-Lex — check the consolidated text of the AI Act for the current application dates before advising on any system.
- Responsible AI framework: NIST (2023) Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1, January 2023, free from nist.gov — the source of the trustworthiness characteristics this module condenses into five principles.
- The UK approach: Department for Science, Innovation and Technology (2023) A pro-innovation approach to AI regulation, white paper, and the government's February 2024 response, both on GOV.UK — the five principles UK regulators apply.
- AI and personal data: Information Commission (the ICO; formerly the Information Commissioner's Office), Guidance on AI and data protection, free at ico.org.uk — practical guidance on lawful basis, fairness and data protection impact assessments for AI.
- AI and copyright: US Copyright Office (2025) Copyright and Artificial Intelligence, Part 2: Copyrightability, January 2025, at copyright.gov; and the UK government's Report on Copyright and Artificial Intelligence (March 2026) on GOV.UK.
- Management system standard: ISO/IEC 42001:2023, Information technology — Artificial intelligence — Management system, from iso.org (paid) — the certifiable structure for organisations that want to formalise AI governance.
- Further practice: run the AI Governance Readiness Audit for your own organisation and keep the result alongside your governance charter: Module 6 uses the charter to build trust, and Module 8 draws on both for its governance commitments.
Did this module make sense?
Thanks for saying so: the Faculty sees every “not really”.
Which part lost you? (optional, anonymous)
Thanks: that tells the Faculty exactly where to look.
In the meantime:
Key terms in this module
- EU AI Act
- The European Union's AI regulation (in force August 2024), which classifies AI systems by risk level and sets mandatory requirements for high-risk applications.
- High-risk AI
- Under the EU AI Act, AI used in areas such as employment, credit scoring, critical infrastructure or biometric identification — subject to strict conformity requirements.
- AI governance
- The policies, roles, processes and controls an organisation puts in place to ensure its AI is developed and used in a legal, ethical and effective way.
- Responsible AI
- A set of principles — fairness, transparency, accountability, privacy, reliability — that guide the design and operation of AI systems.
- Model risk
- The risk that an AI model produces incorrect or biased outputs that lead to poor decisions, financial loss or reputational damage.
- AI-generated content (IP)
- Output produced by an AI system; whether it attracts copyright, and who owns it, depends on jurisdiction and on the level of human creative input.
Thanks. Good to hear.