AI, especially Generative AI, is reshaping financial services, enhancing products, client interactions, and productivity. However, challenges like hallucinations and model unpredictability make safe deployment complex. Rapid advancements require flexible governance.

Financial institutions are eager to adopt AI but face regulatory hurdles. Existing frameworks may not address AI’s unique risks, necessitating an adaptive governance model for safe and compliant integration.

The following framework has been developed by FINOS (Fintech Open Source Foundation) members, providing comprehensive catalogue or risks and associated mitigation. We suggest using our heuristic risk identification framework to determine which risks are most relevant for a given use case.

Risk Catalogue

Operational

AIR-OP-004

Hallucination and Inaccurate Outputs

SummaryLLM hallucinations occur when a model generates confident but incorrect ...

AIR-OP-005

Foundation Model Versioning

SummaryFoundation model instability refers to unpredictable changes in model behavior ...

AIR-OP-006

Non-Deterministic Behaviour

SummaryLLMs exhibit non-deterministic behaviour, meaning they can generate different outputs ...

AIR-OP-007

Availability of Foundational Model

SummaryFoundation models often rely on GPU-heavy infrastructure hosted by third-party ...

AIR-OP-014

Inadequate System Alignment

SummaryAI alignment risk arises when a system’s behaviour diverges from ...

AIR-OP-016

Bias and Discrimination

SummaryAI systems can systematically disadvantage protected groups through biased training ...

AIR-OP-017

Lack of Explainability

SummaryAI systems, particularly those using complex foundation models, often lack ...

AIR-OP-018

Model Overreach / Expanded Use

SummaryModel overreach occurs when AI systems are used beyond their ...

AIR-OP-019

Data Quality and Drift

SummaryGenerative AI systems rely heavily on the quality and freshness ...

AIR-OP-020

Reputational Risk

SummaryAI failures or misuse—especially in customer-facing systems—can quickly escalate into ...

Security

AIR-SEC-002

Information Leaked to Vector Store

SummaryLLM applications pose data leakage risks not only through vector ...

AIR-SEC-008

Tampering With the Foundational Model

SummaryFoundational models provided by third-party SaaS vendors are vulnerable to ...

AIR-SEC-009

Data Poisoning

SummaryData poisoning occurs when adversaries tamper with training or fine-tuning ...

AIR-SEC-010

Prompt Injection

SummaryPrompt injection occurs when attackers craft inputs that manipulate a ...

Regulatory and Compliance

AIR-RC-001

Information Leaked To Hosted Model

SummaryUsing third-party hosted LLMs creates a two-way trust boundary where ...

AIR-RC-022

Regulatory Compliance and Oversight

SummaryAI systems in financial services must comply with the same ...

AIR-RC-023

Intellectual Property (IP) and Copyright

SummaryGenerative AI models may be trained on copyrighted or proprietary ...

Preventative

AIR-PREV-002

Data Filtering From External Knowledge Bases

PurposeThis control addresses the critical need to sanitize, filter, and ...

AIR-PREV-003

User/App/Model Firewalling/Filtering

Effective security for AI systems involves monitoring and filtering interactions ...

AIR-PREV-005

System Acceptance Testing

PurposeSystem Acceptance Testing (SAT) for AI systems is a crucial ...

AIR-PREV-006

Data Quality & Classification/Sensitivity

PurposeThe integrity, security, and effectiveness of any AI system deployed ...

AIR-PREV-007

Legal and Contractual Frameworks for AI Systems

PurposeRobust legal and contractual agreements are essential for governing the ...

AIR-PREV-008

Quality of Service (QoS) and DDoS Prevention for AI Systems

PurposeThe increasing integration of Artificial Intelligence (AI) into financial applications, ...

AIR-PREV-010

AI Model Version Pinning

PurposeModel Version Pinning is the deliberate practice of selecting and ...

AIR-PREV-012

Role-Based Access Control for AI Data

PurposeRole-Based Access Control (RBAC) is a fundamental security mechanism designed ...

AIR-PREV-014

Encryption of AI Data at Rest

PurposeEncryption of data at rest is a fundamental security control ...

AIR-PREV-017

AI Firewall Implementation and Management

PurposeAn AI Firewall is conceptualized as a specialized security system ...

Detective

AIR-DET-001

AI Data Leakage Prevention and Detection

PurposeData Leakage Prevention and Detection (DLP&D) for Artificial Intelligence (AI) ...

AIR-DET-004

AI System Observability

PurposeAI System Observability encompasses the comprehensive collection, analysis, and monitoring ...

AIR-DET-009

AI System Alerting and Denial of Wallet (DoW) / Spend Monitoring

PurposeThe consumption-based pricing models common in AI services (especially cloud-hosted ...

AIR-DET-011

Human Feedback Loop for AI Systems

PurposeA Human Feedback Loop is a critical detective and continuous ...

AIR-DET-013

Providing Citations and Source Traceability for AI-Generated Information

PurposeThis control outlines the practice of designing Artificial Intelligence (AI) ...

AIR-DET-015

Using Large Language Models for Automated Evaluation (LLM-as-a-Judge)

Purpose“LLM-as-a-Judge” (also referred to as LLM-based evaluation) is an emerging ...

AIR-DET-016

Preserving Source Data Access Controls in AI Systems

PurposeThis control addresses the critical requirement that when an Artificial ...