AI in the Financial Sector: Why a Robust AI Risk Management Framework Is Now Essential

Artificial intelligence has evolved from an innovation topic into a business-critical ICT component within the financial sector. AI systems support credit processes, fraud detection, customer service, document analysis, software development, risk modelling, and internal control processes. In addition to customer- and risk-related use cases, AI is increasingly being used in internal finance processes, such as automated accounting procedures and account reconciliations.

AI as a Regulated ICT Asset: Supervisors Are Taking a Closer Look

As AI becomes more deeply embedded in operations, ICT risks increase accordingly. Data poisoning, prompt injection, hallucinations, model manipulation, insufficient explainability, cloud concentration risks, and third-party dependencies are becoming concrete governance and resilience issues. At the same time, AI is attracting increasing regulatory attention because its use can have a direct impact on risk management, operational resilience, and consumer protection.

The key shift in perspective is this: AI is not a special case but an ICT asset with specific risk characteristics. Financial institutions and IT service providers must consistently integrate models, training data, prompts, APIs, vector databases, cloud services, model providers, and monitoring tools into their existing ICT governance frameworks.

The BaFin Guidance on ICT Risks Associated with the Use of AI explicitly confirms this classification. The term “AI system” is understood as a combination of ICT assets and ICT infrastructure; like ICT systems in general, AI systems serve business processes. Therefore, it is not only the model itself that matters, but also its integration into processes, data flows, infrastructure, interfaces, and operating environments. The guidance further emphasizes that specific ICT risks arise not primarily from an AI system’s functional role along the value chain, but from its integration into the ICT landscape and must be assessed throughout the AI lifecycle.

Target Operating Model: AI Risk Management as an Extension of DORA

A robust AI risk management framework starts with clear AI governance. Responsibilities, management involvement, role models, risk appetite, and decision-making processes must be clearly defined. Building on this foundation, financial institutions and IT service providers in the financial sector require a comprehensive AI inventory covering deployed systems, models, providers, data flows, and supported business processes, as well as a risk taxonomy that systematically captures governance, data, model, cyber, operational, and third-party risks.

This approach aligns with the principles set out in the BaFin guidance: AI systems should be managed within the existing ICT risk management framework. In particular, BaFin highlights governance and organisation, identification, protection and prevention, detection, response and recovery, learning processes, and communication as core components of the DORA framework. For those responsible, this means that AI governance should not be established as a parallel regime but rather as an auditable extension of the existing DORA and ICT risk management framework.

Banks, asset management companies, insurers, and other financial institutions must manage the security and resilience of AI systems throughout the entire AI lifecycle, including procurement, development, testing, deployment, operation, and decommissioning. Particular importance should be placed on AI-specific security measures addressing prompt injection, data poisoning, model extraction, adversarial attacks, and unauthorized access, as well as robust third-party risk management covering cloud providers, model providers, platforms, open-source dependencies, and subcontractors. In addition, AI-related incidents must be integrated into monitoring and ICT incident management processes, while governance, controls, documentation, and effectiveness must be reviewed regularly.

Despite Outsourcing: Financial Institutions Remain Responsible for Adequacy and Effectiveness

An AI risk management framework is only robust if it is auditable, thoroughly documented, and demonstrable to clients.

Financial institutions increasingly outsource AI-related controls, data processing activities, cloud platforms, model operations, and security functions to IT service providers. However, from a regulatory perspective, they remain responsible for ensuring the adequacy and effectiveness of outsourced controls. As a result, financial institutions require assurance that AI-specific risks are being effectively managed by the service provider, particularly with regard to governance, access controls, data quality, model changes, monitoring, incident management, and third-party dependencies.

The BaFin guidance also highlights the significance of ICT third-party risk in the context of AI, particularly regarding dependencies on cloud providers, model providers, and platforms. Where AI applications support critical or important functions, service-level agreements, security arrangements, subcontractor transparency, and comprehensive audit and control rights become especially relevant. For IT service providers, this presents an opportunity to build trust and simplify compliance for regulated clients through structured control reporting under IDW PS 951 or ISAE 3402.

Baker Tilly supports IT service providers and financial institutions in systematically establishing this capability to demonstrate compliance. Our approach is tailored to each participant’s role within the AI ecosystem. IT service providers require control-ready processes, documentation, and independent assurance reports, for example under IDW PS 951 or ISAE 3402. Software providers of accounting-relevant applications must additionally address requirements for transparent, tested, and reliable software processing, including finance-related program functions. Where AI functions are integrated into accounting, valuation, reporting, reconciliation, or control processes, Baker Tilly can assist in structuring software controls and documentation with reference to IDW PS 880. For software providers of accounting-relevant applications, a software audit under IDW PS 880 can provide a robust demonstration of trustworthiness and quality. Such an audit confirms that program functions, development and release processes, and controls are suitable for supporting the proper processing of business transactions. This is particularly relevant where upstream process systems function as source systems for financial accounting.

For financial institutions themselves—such as banks, asset management companies, and insurers—the focus lies on embedding specific AI and AI-agent use cases into their existing ICT and DORA risk management frameworks. This is especially relevant for business units that are already using AI agents or planning to do so, as agentic systems can independently retrieve information, prepare decisions, initiate process steps, or orchestrate multiple tools. These capabilities create new requirements regarding role and access management, logging, human oversight, data classification, model and prompt governance, as well as control and escalation mechanisms. Baker Tilly supports organisations through AI risk management workshops, during which use cases, risks, control requirements, and regulatory considerations are assessed in a structured manner and translated into a practical target operating model.

Conclusion

In the financial sector, AI is no longer a regulatory experiment. The central expectation is clear: those responsible must integrate AI systems into their existing DORA and ICT risk management frameworks.

This creates concrete areas for action for different stakeholders. Financial institutions and business units must integrate AI and AI-agent use cases into their DORA and ICT risk management frameworks in a controlled manner. IT service providers can build trust with regulated clients through robust assurance reporting under IDW PS 951 or ISAE 3402. Software providers of accounting-relevant applications should additionally ensure that their software controls are demonstrably effective, with reference to IDW PS 880. The key is not to view AI risk management as additional bureaucracy, but rather as a prerequisite for scalable, secure, and regulatorily resilient use of artificial intelligence.