Summary

• AI Model Risk Management is essential for organisations looking to integrate AI into existing processes or build bespoke machine learning systems.
• AI Model Risk Management will ensure transparency and regulatory/ethical compliance. It will also help all users understand the scope of what the system can achieve.
• Clear corporate governance, strategy, communication, and documentation are needed to successfully build and AI Model Risk Management policy.
• Instructing a Risk Management Specialist to create your AI Model Risk Management Policy frees up internal resources and provides an objective, holistic view of the associated risks.

AI or ML (machine learning) Model Risk Management (MRM) gives an organisation a clear, practical risk management plan when developing and rolling-out AI or ML models. Having a robust MRM helps foster a culture of responsibility, caution, and trust within an organisation looking to harness this innovative technology so that all stakeholders can safely enjoy the benefits.

The increase in AI and ML models and the opportunities they continue to present to all organisations means many companies are significantly investing in this technology. A report commissioned by technology consulting company, Searce in Q3 2024, found that  8% of UK decision-makers planned to spend over £19.5 million on AI initiatives over the next 12 months. The top reason cited for investing in AI was new business growth and 90% of the 300 C-suite executives surveyed view their AI initiatives as “successful”.

If you are riding this change that is tipped to be the biggest economic shake-up since the industrial revolution, you must have robust risk management in place to protect the interests of not only your business, but your customers/clients, employees, directors, and investors/shareholders.

Why is AI Model Risk Management Important?

No new technology is free from risk. This is especially true in relation to complex ML systems. The following are common risks associated with AI and ML platforms:

• Issues with data quality – any AI or ML model is only as good as the data that is fed into it. So, it follows that if the inputted data is flawed or bias, there is a serious risk of incorrect and/or prejudice data being included in the output.
• Hallucinations – ever since Google’s AI Overviews told searchers they should eat rocks, the issue of AI ‘hallucinations’ has been an overriding concern. In the context of Large Language Models (LLMs), “hallucination” refers to a phenomenon where the model generates text that is incorrect, nonsensical, or simply made up. Unfortunately, hallucinations are largely unavoidable; therefore, the risk of their occurrences must be factored in and users informed that the information they retrieve could be inaccurate.
• Choosing an incorrect design or AI model – if you fail to implement or develop the right model architecture, there is a significant risk your innovation will not work as well as it should or provide misleading or incorrect information. This is especially true when using open source models (where the software’s source code is publicly available on the internet).
• Out of date training material – LLMs are traditionally trained on historical data that can quickly become unreliable, especially when it comes to understanding customer, economic, or social trends. For example, it has become clear that children who were born during the Coronavirus pandemic suffered from poor early language development. If educators are using an AI or LLM model to distil and analyse data related to early childhood educational needs, and they are unaware that the tool they’re using has not been fed post-pandemic training data, they could base important policies on results that don’t reflect the true needs of some children.

What are the components of an AI Model Risk Management framework?

An MRM framework provides risk analysts with a structured process to identify and assess model-specific risks. It is built on several components, including:

• Vigorous standards relating to development, testing, data protection, and regulatory compliance frameworks.
• Clear corporate governance regarding the objectives of using AI and LLMs within the organisation and safeguards in place to ensure they are effectively managed and align with the business’s overall strategy.
• Continuous model monitoring and testing to ensure any inaccuracies, bias, and hallucinations are spotted and fixed where possible.
• A constant strategy and protocol for communicating the limitations of the model.
• A detailed risk register that has been compiled from the information gleaned from analysing the risk scope, risk identification, risk assessment, and risk mitigation considerations of the model.
• Comprehensive documentation and regular reporting relating to the initial risk management exercise and ongoing risk assessments.

What is an example of an AI Model Risk Management Framework?

An MRM framework can comprise of:

• Mode Cards – these set out the model’s purpose, training data, capabilities, limitations, and performance They ensure users have a clear understanding of the scope of the system’s capabilities.
• Data Sheets – outline the dataset used to train an ML model. In addition, they set out the creation process, composition (data types, formats), intended uses, potential biases, limitations, and any ethical and regulatory considerations associated with the data.
• Risk Cards – these provide details on the key risks of the model and how they will be managed.
• Scenario Planning: Involves testing using hypothetical circumstances in which the model could be misused or crash in order to spot unforeseen risks and build mitigation tactics.

Concluding comments

Establishing a successful AI model risk management framework is a resource-hungry commitment. Having an independent advisor to manage the process allows your company’s resources to be directed to other profit-making areas. Furthermore, your organisation will benefit from having an objective, holistic view of the risks associated with the technology.

At 43Legal, we have the knowledge and resources to undertake a comprehensive risk management process. We can also advise and represent you if a dispute develops. We will resolve the dispute quickly and cost-effectively while protecting your best interests.

To learn more about any matters discussed in this article, please email us at info@43legal.com or phone 0121 249 2400.

The content of this article is for general information only.  It is not, and should not be taken as, legal advice.  If you require any further information in relation to this article, please contact 43Legal.

Melissa Danks is the founder of 43Legal. She has over 20 years’ experience as a solicitor working within the legal sector dealing with issues relating to risk management, dispute resolution, and advising in-house counsel in SMEs and large companies. Melissa has extensive expertise in providing practical, valuable, modern legal advice on large commercial projects, joint ventures, data protection and GDPR compliance, franchises, and commercial contracts. She has worked with stakeholders in multiple market sectors, including IT, legal, manufacturing, retail, hospitality, logistics and construction. When not providing legal advice and growing her law firm, Melissa spends her time running, walking in the countryside, reading and enjoying downtime with close friends and family.