AI in banking: use-case governance determines whether AI chat tools pay off

3D-Rendering: Gläserne, durchsichtige Skulpturen in Lila-/Flieder-Tönen. Darin ist Computercode zu sehen.
  • 11/19/2026
  • Reading time 7 Minutes

AI chat tools do not deliver efficiency gains on their own. Banks realize economic value only when they manage usage, costs, model selection, and use cases transparently and direct them toward applications that create value.

Many banks are currently weighing whether and how to deploy artificial intelligence, and whether it can deliver on its promised efficiency gains. To find an answer, many institutions start by introducing generative AI through an internal chat tool. These tools typically run on external large language models (LLMs) such as ChatGPT, Gemini, or Claude. They help structure knowledge and text and are designed to give employees an easy entry point into AI.

The expected efficiency gains do not always materialize automatically, however. Without transparency over usage, costs, and suitable use cases, deployment can quickly create additional work and reinforce the perception that generative AI does not make economic sense. What matters is not simply providing the technology, but managing specific use cases in a targeted way.

Artificial intelligence and the efficiency question: are chat tools the start or the end of the journey?

External providers typically bill their LLMs on a usage basis, by input and output tokens. Tokens are word fragments or character strings of one to four characters that serve as the LLM’s input and output: the prompts users enter into the chat and the model’s responses. This creates variable costs that depend heavily on user behavior, the complexity of the query, and the model used. While some providers currently do not bill variably by token consumption, the business market shows a clear trend toward passing token costs on to customers on a usage basis.

The use case therefore determines whether a positive business case emerges. Simple knowledge queries can deliver noticeable time savings at low cost. With iterative email revisions, by contrast, the main benefit may be higher quality rather than saved working time. The choice of model is also decisive. The latest top-tier models cost almost twice as much as older or “simpler” models. Banks should therefore select the most cost-effective model on the basis of the use case. Older models still do the job and handle comparatively simple tasks, such as knowledge queries or text editing, comparably well.

Not every use of an LLM produces a positive business case

The following three examples show that actual usage determines whether an LLM chat tool is economically viable:

Example: simple knowledge queries

Employees have traditionally answered simple questions such as “Where can I find the work instruction for releasing collateral on a repaid loan?” or “Which documents does our checklist require to open an account for a GmbH in formation?” by looking them up, asking a colleague, or searching online. This takes working time, for example to review search results manually.

An LLM chat tool reduces this effort. The input usually consists of only a few tokens, while the output can be many times larger depending on the question (up to 20 times). With many providers, a single query costs in the low single-digit cents.

These variable costs can add up quickly and significantly as usage grows. Overall, however, the time saved through AI outweighs them. The savings from recovered employee time exceed the cost of the tool.

Example: emails

Drafting and quality-checking emails takes working time. An LLM chat tool can noticeably improve quality and support the writing process. Yet the expected savings in time, and therefore in cost, often fail to materialize:

A user drafts an email, has the LLM chat tool improve it, and runs through several iterations until reaching the desired result. As a recipient the user does the same. Additionally, the incoming email is used as an input when generating a reply. The result is a relatively balanced ratio of input to output tokens (40/60), which costs significantly more than a simple knowledge query (up to five times as much, according to a Baker Tilly analysis).

In this example, both users end up with better emails, but the revision loops save no working time. The additional usage costs are offset by a gain in quality, whose value must be assessed against the purpose of the communication. The lack of time savings alone would point to a negative business case, which must be weighed against the quality gain case by case.

Example: document analysis

Analyzing regulatory documents, whether a new EU directive, a new amendment to the MaRisk (BaFin’s Minimum Requirements for Risk Management), or EBA Guidelines, takes working time. An initial review of the changes in an updated guideline alone can take several hours.

An LLM chat tool can significantly accelerate this first step and substantially cut the working time it requires. Token costs weigh against this: inputs and outputs for tasks of this kind quickly run into six-figure token counts and create a far larger individual cost than the two previous examples (up to 40 times as much, according to a Baker Tilly analysis). The reason is that the input includes not only the question but also the regulatory document itself, which is often very extensive and therefore drives costs. The AI will also ask follow-up questions to further structure the research.

Overall, the business case is positive, thanks to the time saved on an initial summary and on identifying relevant passages. Targeted use of an LLM chat tool therefore saves time and reduces costs. The work moves more quickly to interpretation, regulatory assessment, and, ultimately, implementation.

A central use-case register creates transparency and makes LLM costs manageable

A use case becomes inefficient or negative whenever the time or cost savings are small, or when users deploy AI chat tools poorly (for example, through overly long chat histories, excessively large attachments, or vague prompting). Broad use of LLMs therefore does not in itself guarantee economic success. That requires transparent, use-case-based governance.

To manage their budgets, banks must first create transparency over who is using the variably billed models. One way to do this is to make each user’s actual token consumption visible.

Another is to establish a central use-case register that assigns each use case, for example, a business purpose, a cost figure based on the model used, and a cost figure reflecting the time saved. Costs can then be managed effectively by reference to a cost-benefit ratio. On this basis, banks can also set a use-case budget for each business area.

Model routing is a further management tool: it selects the appropriate AI model for each use case. A top-tier model would then be used not for a simple knowledge question, but only for complex research such as the analysis of extensive documents.

Conclusion: AI chat tools mark the start of efficient AI implementation only if their rollout is deliberately managed

Introducing an AI-powered chat tool is a good way to familiarize employees with AI. A bank’s medium- and long-term AI strategy, however, should rest on effective use-case governance.

A central use-case register is the key management instrument. It creates transparency over the purpose, costs, model choice, and expected benefits of individual applications. It sets out the costs and benefits of AI solutions clearly and lays the foundation for strategic decisions. By combining the register with cost-benefit assessments, use-case budgets, and model routing, institutions can direct the use of their tools to where they genuinely create efficiency.

Rolling out an internal LLM chat tool is therefore not the goal. It is the starting point for bank-specific GenAI applications that are cost-effective and widely accepted.

We would be pleased to help you assess the cost-effectiveness and controllability of your AI tools and to advance the rollout of bank-specific GenAI applications with a clear focus.