AI, especially Generative AI, is reshaping financial services, enhancing products, client interactions, and productivity. However, challenges like hallucinations and model unpredictability make safe deployment complex. Rapid advancements require flexible governance.

Financial institutions are eager to adopt AI but face regulatory hurdles. Existing frameworks may not address AI’s unique risks, necessitating an adaptive governance model for safe and compliant integration.

The following framework has been developed by FINOS (Fintech Open Source Foundation) members, providing comprehensive catalogue or risks and associated mitigation. We suggest using our heuristic risk identification framework to determine which risks are most relevant for a given use case.

Risk Catalogue

Operational

AIR-OP-004

Hallucination and Inaccurate Outputs

LLM hallucinations refer to instances when a large language model ...

AIR-OP-005

Instability in foundation model behaviour

Instability in foundation model behaviour would manifest itself as deviations ...

AIR-OP-006

Non-deterministic behaviour

A fundamental property of LLMs is the non-determinism of their ...

AIR-OP-007

Availability of foundational model

RAG systems are proliferating due to the low barrier of ...

AIR-OP-011

Lack of foundation model versioning

Inadequate or unpublished API versioning and/or model version control may ...

AIR-OP-014

Inadequate system alignment

AlignmentThere is a specific goal you want to achieve when ...

AIR-OP-016

Bias and Discrimination

AI trained on historical/internet data may embed biases. Can lead ...

AIR-OP-017

Lack of Explainabililty

Black Box Nature of Generative Models Difficult to interpret and ...

AIR-OP-018

Model Overreach & Misuse

The impressive capabilities of GenAI can lead to overestimation of ...

AIR-OP-019

Data Quality & Drift

Generative AI’s outputs depend on the quality and recency of ...

AIR-OP-020

Reputational Risk

AI failures or misuse can quickly become public incidents, eroding ...

Security

AIR-SEC-002

Unauthorized Access and Data Leaks

TODO: Make this non-vector store specificVector stores are specialized databases ...

AIR-SEC-008

Tampering with the foundational model

The SaaS-based LLM provider is a 3rd party supplier and ...

AIR-SEC-009

Data Poisoning

Adversaries can tamper with AI training or fine-tuning data to ...

AIR-SEC-010

Prompt injection

Users of the application or malitious internal agents can craft ...

Regulatory and Compliance

AIR-RC-001

Information Leaked to Hosted Model

In the provided system architecture, sensitive data is transmitted to ...

AIR-RC-022

Regulatory Compliance and Oversight

Financial services are heavily regulated, and AI use does not ...

AIR-RC-023

Intellectual Property (IP) and Copyright

Generative AI models often train on datasets that may include ...