ICT risks when using AI: New BaFin guidance

The financial industry is on the cusp of one of the most profound changes in its history. The integration of artificial intelligence (AI) into core banking processes promises efficiency gains, more accurate risk models, and hyper-personalized customer interactions.

But with technological power comes vulnerability. The German Federal Financial Supervisory Authority (BaFin) has sent a clear signal with its “Guidance on ICT risks when using AI in financial companies”: the era of unregulated experimentation is over. AI is no longer an abstract topic for the future, but a concrete ICT asset that is also subject to the strict requirements of the Digital Operational Resilience Act (DORA).

This article summarizes key aspects of the guidance published on December 18, 2025. We begin by illustrating modern, AI-supported lending in order to explain the operational risks involved. We then analyze BaFin's requirements throughout the entire AI lifecycle—from governance and development to decommissioning. Finally, we will outline the contribution independent auditing can make to obtaining reliable evidence of prudent risk management with AI risks and operational resilience.

The Anatomy of Decision-Making – How AI is Revolutionizing Lending

In order to fully understand the regulatory requirements of BaFin, it is essential to clarify what these rules actually apply to.

The paradigm shift: From causality to correlation

Traditional lending processes were based on causal relationships and linear rules according to the formula: “If income > X and debt < Y, then loan = yes.” This deterministic worldview is being fundamentally challenged by AI.

Modern systems, often based on machine learning (ML), do not primarily search for causalities, but rather for complex, non-linear correlations in huge data sets. They transform the question “Can the customer pay?” into a statistical probability forecast that weighs a multitude of variables (features) against each other in a fraction of a second.

The technical workflow: A deep dive analysis

The life cycle of an AI-based lending decision can be technically divided into five critical phases. Each of these phases involves specific ICT risks, which are addressed in the BaFin guidance.

Phase 1: Intelligent data capture and extraction (input management)

The process begins at the point of sale – whether in an app or at the bank counter.

• Technology: This area is dominated by Intelligent Document Processing (IDP). Instead of manual data entry, Optical Character Recognition (OCR) and Natural Language Processing (NLP) are used.
• Process: The customer uploads unstructured documents (PDF pay slips, JPG images of ID cards, bank statements). AI agents first classify the document (e.g., “This is a 2024 tax assessment notice”) and then extract specific entities (such as EBITDA, IBAN, tax class).
• Risk implication: This is where the risk of “data poisoning” or, more simply, extraction errors arises. If the AI model misinterprets a digit, the entire downstream risk calculation is flawed (“garbage in, garbage out”).

Phase 2: Feature engineering and data enrichment

The raw data is now transformed into processable signals.

• Alternative data sources: Modern AI systems are not limited to data from the German General Credit Protection Agency (“Schufa”). They integrate external data points such as transaction histories (open banking), geolocation data, social media sentiment, or even typing behavior when filling out the application (behavioral biometrics) via application programming interfaces (APIs).
• Feature Creation: Algorithms calculate derived variables. Example: Instead of just looking at the account balance, the AI analyzes the account balance’s volatility over 24 months or the ratio of gambling expenses to fixed costs.

Phase 3: Risk prediction (inference)

This is the heart of the system – the “black box.”

• Modeling: Once dominated by logistic regression, banks now rely on gradient boosting machines (e.g., XGBoost, LightGBM) or neural networks. These models are capable of recognizing complex interactions between variables (e.g., that a high income combined with high volatility in a particular industry represents a higher risk than previously assumed).
• Prediction: The model does not generate a binary output (yes/no), but rather a probability of default (PD) or a score (e.g., 0.0 to 1.0).

Phase 4: Decision-making and explainability (XAI)

A score alone does not meet regulatory requirements.

• Explainable AI (XAI): Methods such as SHAP (SHapley Additive exPlanations) are used to comply with regulatory transparency requirements and the prohibition of automated decisions without explanation (GDPR). They calculate each individual characteristic’s contribution to the final result (e.g., “Income increased the score by 0.2, credit history decreased it by 0.4”).
• Human-in-the-loop: In borderline cases or high volumes, the system forwards the case to a human analyst (“referral”). In this case, AI acts as “augmented intelligence,” providing humans with a pre-analyzed decision template.

Phase 5: Continuous monitoring and fraud detection

The AI lifecycle does not end after the initial credit decision. Rather, a permanent operational and monitoring phase begins which, in terms of ICT risks, is one of the most critical phases from a regulatory perspective.

• Fraud detection:
  In parallel with credit checks, specialized AI models are used for fraud detection. Unsupervised learning methods analyze applications and transactions for deviations from established patterns, such as implausible IP addresses, conspicuous metadata in documents, or unusual application histories. The aim is to detect new fraud patterns at an early stage that are not captured by rule-based systems.
• Model and data monitoring:
  AI models are subject to the risk of data and model drift. Changes in customer behavior, the economic situation, or regulatory conditions can cause originally valid models to systematically deliver incorrect results. Financial institutions therefore continuously monitor input data, model outputs, and performance indicators, and define thresholds and escalation mechanisms.
• Self-learning and model changes:
  If models are regularly retrained or automatically adapt to new data, this can make it considerably more difficult to track and control system behavior. Model changes therefore generally require appropriate controls. Depending on the nature of the model, these can include formalized approval, testing, and documentation processes as well as ongoing monitoring of system behavior, for example, through defined thresholds, plausibility checks, or performance monitoring. Ex-ante approval takes a back seat, especially in the case of automatically adjusting models; continuous monitoring and documentation of model effects are of central importance in order to identify and limit undesirable effects or regulatory violations at an early stage.
• Risk implication:
  A lack of or insufficient monitoring can lead to AI systems making incorrect or inexplicable decisions over a longer period of time. Phase 5 is therefore the central control point that determines whether the use of AI remains stable, transparent, and controllable in terms of regulatory requirements in the long term.

The need for regulation

This massive automation and dependence on complex, often nontransparent algorithms create new vulnerabilities. What happens if the model has been trained on discriminatory historical data (bias)? What if attackers deceive the model by making minimal changes to the input data (adversarial attacks)? What if the model suddenly delivers incorrect forecasts due to changes in the macroeconomic environment (e.g., pandemic, inflation) (model drift)? This is precisely where BaFin comes in with its guidance. It does not view the AI system as a magic crystal ball, but as a critical ICT asset that must be managed, secured, and monitored.

The BaFin guidance

The guidance is based, among other things, on discussions with financial companies and does not represent a binding interpretation of DORA by BaFin. Overall, the supervisory authority makes it clear that the handling of AI risks is under observation. Financial companies deviating from the guidance expose themselves not only to ICT but also to compliance risks. In the following, we analyze the document chapter by chapter in order to illustrate the depth of the requirements.

Introduction and contextualization

Legal nature

BaFin makes it clear that this is “non-binding guidance.” In financial supervision practice, however, this regularly means a reversal of the burden of proof. Anyone who ignores the guidance must provide detailed evidence in the event of an audit that their alternative measures offer at least an equivalent level of protection. The guidance is primarily aimed at CRR institutions (credit institutions) and Solvency II insurers that are required to apply the full ICT risk management framework under Articles 5-15 of DORA.

Definition of an AI system

BaFin is not reinventing the wheel, but refers to the definitions in the EU AI Regulation (AI Act). However, the classification as a “machine-assisted system” is crucial for IT supervision. This definition firmly anchors AI in the concept of “network and information systems” according to DORA. This means that all general DORA requirements automatically also apply to AI systems – supplemented by AI-specific risks, such as stochastics.

ICT risk management

Governance and strategy

If AI applications support critical or important functions, the guidance recommends that an AI strategy be formulated. This can be a standalone strategy or integrated into the IT/DORA strategy. The strategy must clarify why AI is being used (efficiency, risk minimization), what risks are acceptable (risk appetite), and what the resource planning looks like. A strategy that calls for AI innovation but does not provide budgets for cloud infrastructure or specialized personnel is inconsistent.

BaFin emphasizes the management's ultimate responsibility (Art. 5 (2) DORA). Board members cannot claim ignorance. DORA explicitly requires that members of the management body acquire sufficient ICT knowledge. In the context of AI, this means that while a bank board member does not need to be able to write code, they must understand what “model drift” is, why “hallucinations” in LLMs pose a risk, and where the limits of automation lie.

Integration into the risk management framework

AI risks must not be viewed in isolation, financial regulators demand in their guidance. Financial companies must conduct a complete inventory of their AI systems. This includes “shadow AI” and AI components in purchased standard software (e.g., HR tools, ticketing systems).

Furthermore, BaFin stipulates that risk treatment measures must be specific, i.e., they must address a specific risk. If a risk has been identified in the area of “adversarial attack,” the measure must be technical in nature (e.g., adversarial training) and not merely organizational.

Provision of AI: Development and testing

Here, BaFin delves into the technical implementation and applies principles of software development to the discipline of data science.

Software development

BaFin considers the training of a model similar to the compilation of software.

• One ICT risk in many financial companies is “shadow IT” in specialist departments, where actuaries or risk analysts develop complex models in Python or R on their laptops. BaFin makes it unmistakably clear that this EUC is also subject to the full requirements for development processes, versioning, and documentation.
• The use of open-source libraries (TensorFlow, PyTorch, Hugging Face) is standard practice, but carries outsourcing risks. BaFin emphasizes static code analysis to ensure that no malicious code has been introduced and to identify “hidden” AI functionalities in third-party software.

The testing paradigm

Testing stochastic systems (which are based on probabilities) is fundamentally different from testing deterministic software.

• BaFin explicitly recommends “attacking” AI systems (adversarial testing): How does the model react when 1 % of the training data is manipulated (data poisoning tests)? Can the model be deceived by noise in the input data (evasion attacks)?
• How does the model behave under extreme market conditions that did not occur in the training data (e.g., a crash in the bond market)? Such scenarios must be simulated (stress tests).

Operation and decommissioning of AI

An AI model is not a “fire and forget” system. It ages from the moment it is put into operation.

Monitoring and drift

Reality is constantly changing, but the trained model remains static. This discrepancy is called model drift.

• Concept drift: The relationship between the data changes (e.g., inflation means that an income of €50,000 has less purchasing power today than it did five years ago).
• Data drift: The distribution of the input data changes (e.g., suddenly, much younger customers are applying for loans).
• Requirement: BaFin requires continuous monitoring of model drift, e.g., with defined thresholds. If the forecast quality falls below a value X, an alarm must be triggered automatically and, if necessary, retraining must be initiated.
• Logging: Logging must be detailed enough to allow every decision to be reconstructed. Which model in which version made the decision based on which data? This is crucial for post-incident forensics.

Cloud specifics and exit strategies

Since modern AI (especially GenAI) requires scalable computing power, there is often no alternative to the cloud. With regard to vendor lock-in, BaFin warns against dependence on proprietary AI services (e.g., use of AutoML features from a hyperscaler).

Financial companies must develop strategies for maintaining AI operations in the event the cloud provider fails or terminates the contract. This includes the technical capability to port data and models. With proprietary models (such as GPT-4), model export is generally impossible. In this case, functional alternatives (e.g., fallback to an open-source model) must be planned. In addition, BaFin emphasizes that the supervisory notice on outsourcing to cloud providers must be observed.

Cyber and data security

AI systems are attractive targets for cybercriminals, partly because they work with sensitive and valuable data and are also involved in decision-making.

Specific attack vectors
BaFin therefore calls for protective measures against AI-specific attacks:

• Model inversion/extraction: Attackers attempt to copy the model or draw conclusions about sensitive training data by making targeted requests. Countermeasure: Rate limiting and anomaly detection for API access.
• Network security: Training environments should be strictly segmented from production environments and the office network.

Data security

This is where DORA and GDPR intersect. The integrity and confidentiality of data flows must be guaranteed. Data must not only be encrypted “at rest” and “in transit.” BaFin also refers to protection “in use,” which suggests the use of confidential computing (encryption in memory/processor), especially in cloud environments.

How we support you

This article has made it clear that the use of AI is under close scrutiny by companies supervised by BaFin. The requirements are complex, technically profound, and organizationally far-reaching. An error in implementation or documentation can not only result in regulatory sanctions, but also jeopardize the operational resilience of your company.

Our methodology

We are an interdisciplinary team of cyber and IT control experts who are familiar with regulatory requirements. We audit whether your use of the AI system meets the requirements of the BaFin guidance. The result of such audit will be presented to you in a comprehensive and easy-to-understand report that explains all aspects of the guidance throughout the AI lifecycle.

The audit is conducted in accordance with IDW PS 860 (“IT Audit Outside the Scope of the Annual Audit”). This standard of the Institute of Public Auditors in Germany (IDW) is specifically designed to subject IT-supported systems, processes, or applications to an objective assessment separate from the annual audit. It provides the ideal methodological framework for AI audits, as it can be flexibly applied to specific criteria catalogs (such as DORA and the BaFin guidance).

Benefits of the audit for your company

1. Independent confirmation: You receive an objective third-party assessment of whether you have implemented a robust risk management framework that prudently manages AI-specific risks. This strengthens your position in relation to other auditing bodies and external supervisory authorities (BaFin/ECB).
2. Comprehensible and meaningful report: We translate technical findings into risks for business operations. Our report clearly shows whether the requirements from the guidance have been implemented. It serves as ideal proof of the management body's fulfillment of its due diligence obligations.
3. Identification of vulnerabilities: By looking at things from an outside perspective, we uncover “blind spots,” for example, in the area of shadow AI (EUC) or dependencies on third-party providers, before they lead to an operational incident.
4. Market confidence: At a time when confidence in AI decisions is fragile, a certified AI system signals to your customers, partners, and investors that you are committed to innovation, but never at the expense of security.

Of course, we are also happy to support our clients outside of a formal audit in accordance with IDW PS 860. Possible approaches range from compact initial and maturity assessments to topic-specific reviews (e.g., governance, risk management, data quality, model control, or control concepts) to technical evaluations of individual AI applications. These formats are particularly suitable for internal classification of the current implementation status, targeted preparation for regulatory audits, or step-by-step further development of your AI governance throughout the life cycle. Please feel free to contact us for more information.