EU AI Act Summary: AI regulation presents companies with new challenges

The legally compliant assignment of AI systems to one of the risk categories is relevant in the context of the AI Regulation and the AI Liability Directive. Accordingly, an AI system is a machine-based system that is designed to operate with varying degrees of autonomy and can demonstrate adaptability after use – for explicit or implicit goals from the input it receives.
AI systems entailing an unacceptable risk are prohibited as they pose a threat to people. These include biometric real-time remote identification systems and AI systems that enable social scoring or manipulation techniques.

Furthermore, there are also high-risk AI systems. These pose a high risk to the health and safety or fundamental rights of individuals, but are permitted. Their development and use are therefore subject to comprehensive documentation, monitoring and quality requirements. AI systems posing a limited risk to the end user are those that are intended for interaction with individuals. They are subject to a limited number of transparency obligations. AI systems that are considered to pose a minimal risk are permitted without further ado.

High-risk AI systems are divided into two categories: On the one hand, this includes AI systems that are used in products subject to EU product safety regulations. On the other hand, it also includes AI systems that are used in sensitive areas such as health, transport, justice or the police and which must be registered in an EU database. In the case of high-risk AI systems, there is a particular obligation to establish a risk management system, record-keeping and transparency, the creation of technical documentation, the obligation for human supervision and the obligation for these AI systems to achieve an appropriate level of accuracy, robustness and cybersecurity.

AI Governance

A central aspect of this is AI governance which describes the entirety of measures that ensure the ethical and legal use of artificial intelligence. Transparency, fairness and compliance with data protection regulations are crucial in this respect.

For companies, this means that the functioning of their AI systems must be comprehensible in order to create trust among users and authorities. In addition, possible distortions in the algorithms must be actively prevented and security gaps avoided. This can only be achieved through regular employee training and clear guidelines. The right governance therefore plays a key role in ensuring that the benefits of AI can be exploited while risks remain controlled.

GPAI systems (General Purpose AI) are AI systems with a general purpose. Providers of GPAI systems must fulfill special requirements, such as the creation and updating of information and documentation as well as compliance with copyright law. GPAI systems are accessible to the general risk classification and are subject to the corresponding requirements.

According to the AI Regulation, AI systems,

• which are intended to be used for the recruitment or selection of individuals (e.g., analysis and filtering of job applications, assessment of applicants)
• which are intended to make decisions that affect, in particular, the termination and conditions of employment 

are to be classified as high-risk AI systems. In this respect, the user or operator of AI systems, as well as providers, importers, dealers or manufacturers of AI systems, are subject to special obligations.

Co-determination rights of the works council

The employer is entitled to determine whether or not AI will be introduced in the employment relationship. However, the co-determination rights of the works council must be taken into account when determining the “how”. In particular, Art. 87 (1) No. 6 BetrVG (German Works Constitution Act) provides for a right of co-determination in the introduction and use of technical equipment that is objectively suitable for monitoring the behavior or performance of employees. 

If the works council has to assess the introduction and use of AI in order to perform its duties, it has the right to consult an expert. The employer must inform the works council about the planned use of AI as early as possible. 

Discrimination

Using AI systems – just as human decisions – may entail discrimination. This is due to the fact that AI applications are fed by humans with training data that may already be unrepresentative and can therefore lead to unequal treatment. But how can this be countered legally? The German General Equal Treatment Act (“AGG”) also offers protection in this context, according to which employees may not be discriminated against either directly or indirectly on the basis of the discrimination criteria listed therein. Due to the technology-neutral wording of the AGG, this may also cover decisions made by AI applications. In the event of discrimination, the disadvantaged party is therefore entitled to compensation and damages under the AGG. 

Requirement to perform the work in person

Employees are already using AI applications at work, such as DeepL for translating or Copilot / ChatGPT for composing texts. However, employees must perform their work personally, i.e., they are generally not allowed to delegate their work to third parties. It is therefore questionable whether the delegation of work tasks to an AI application constitutes the impermissible use of an auxiliary person or the permissible use of an auxiliary tool. 

The decisive factor in this respect is likely to be that the work result is sufficiently checked by the employee for errors and is not passed off unchecked as their own work product. It is therefore advisable to draw up internal guidelines for dealing with AI applications and to train employees in this regard.