idw – Informationsdienst Wissenschaft
Nachrichten, Termine, Experten
Nachrichten, Termine, Experten
Making AI legally compliant
FIS research programme: Tracking down AI fraud
With the signing of the contract on 9 December 2025, the Agentur für Innovation in der Cybersicherheit GmbH (Cyberagentur) is launching the multi-year research programme "Forensics of Intelligent Systems (FIS)". Three selected contractors are starting their work to develop new methods so that intelligent systems can be analysed and checked for manipulation in a manner that is admissible in court.
An AI-supported application system makes automated pre-selections by prioritising CVs according to specific characteristics. But what happens if manipulation of the training data ensures that applicants with certain backgrounds are systematically disadvantaged? Or if the system suddenly decides significantly different outcomes after an update? The black box logic of many AI applications makes such effects difficult to trace – with potentially far-reaching consequences for companies, society and the rule of law.
This is where FIS comes in. Digital forensics provides the methodology for securing, analysing and evaluating the digital traces of intelligently networked systems in a legally compliant manner. However, standards, tools and procedures are still lacking to reliably detect and prove manipulation of changing, learning AI systems. The research programme "Forensics of Intelligent Systems (FIS)" aims to close this gap and create a completely new quality of digital investigative capability.
With FIS, the Agentur für Innovation in der Cybersicherheit GmbH (Cyberagentur) is bringing together three groups composed of renowned research institutions, industry partners and legal expertise. The aim is to develop procedures and tools that make the behaviour of AI systems transparent, reconstructable and verifiable in a reliable manner.
Today, intelligent systems decide about security, mobility and social processes. With FIS, we are creating a research basis for the first time that makes it possible to design AI that is not only powerful, but also forensically verifiable, legally compliant and thus trustworthy," says Prof. Dr Christian Hummert, Scientific Director at the Cyberagentur.
It was therefore important to the researchers at the Cyberagentur to commission various approaches to the research programme and to finance their work:
INSIDE – Intelligent Text and Audio Processing
The research project "INtelligent Security Inspection of Data using forensic Extraction and Analysis (INSIDE)" pursues the aim of laying the foundations for the forensic auditability ("forensic readiness") of future AI systems. The focus is on the development and in- e testing of innovative forensic methods to systematically, comprehensibly and legally reliably detect manipulation, attacks and anomalies in text-based AI applications – particularly in the field of cyber security and in the analysis of network and text data.
INSIDE addresses both technical and legal challenges: forensic procedures are being developed that not only reflect the current state of the art, but also meet the requirements for evidence and compliance in accordance with applicable EU regulations (e.g. AI Act).
The aim is to strengthen the traceability, integrity and transparency of AI systems, thereby creating a robust basis for technical sovereignty, legal certainty and trustworthy artificial intelligence in the digital space.
The project brings together leading experts in artificial intelligence, cyber security, digital forensics and IT law from the Technical University of Mannheim, the Institute for Security and Safety, the University of the Federal Armed Forces in Munich, the University of Twente, cyberintelligence.institute GmbH (CII) and the FZI Research Centre for Information Technology.
EMAS-AI – Intelligent image and video processing
The consortium led by Atos, the Fraunhofer Institute for Applied and Integrated Security AISEC, the Fraunhofer Institute for Intelligent Analysis and Information Systems IAIS, the Institute for Internet Security at the Westphalian University of Applied Sciences and the University of Cologne is working on the EMAS-AI project to develop legally compliant methods for continuously learning image and video AI systems.
The research project, for which the participating parties have been qualified since 2023 with a preliminary short concept and long concept, is developing prototypical methods and tools that will enable legally admissible detection of manipulations in continuously learning AI systems. To this end, a simulation environment is being set up in a research laboratory where software for forensic methods can be tested. Existing AI models will be subjected to simulated attacks to investigate which legally usable traces can be secured. In addition, AI algorithms will be adapted to enable early detection.
FORAImobility – Forensics of autonomous systems
ZF Friedrichshafen AG is leading the FORAImobility consortium in collaboration with the "Law and Digitalisation" research initiative of the Centre for Human | Data | Society at the University of Konstanz. The aim is to develop innovative methods for detecting manipulation and attacks on AI-based autonomous systems in the field of mobility and to be able to provide legally admissible evidence.
Autonomous systems such as self-driving vehicles are highly complex and will continue to learn in the future. To this end, they must be securely protected against novel cyber attacks. The project addresses these challenges by developing concepts for "forensic- -readiness" and "forensic-by-design," i.e., the ability to integrate forensic evidence into the system architecture from the outset.
ZF is contributing its comprehensive expertise in the areas of cybersecurity, IT forensics and neural networks and is working closely with the "Law and Digitalisation" research initiative of the Centre for Human | Data | Society at the University of Konstanz. The aim is to jointly develop solutions that are both technologically and legally robust. The research programme will run over several years and is of great social and technological relevance.
A glimpse into the future: Trustworthy AI as an infrastructure principle
Whether in application processes, telemedicine or networked traffic flows, artificial intelligence will be deeply integrated into everyday processes in the coming years. The results of the FIS programme are intended to ensure that these systems remain explainable, resistant to manipulation and, in serious cases, assessable under the rule of law. FIS is laying the foundations to ensure that AI systems not only function, but also earn the trust of a free and digital society.
Further information:
https://www.cyberagentur.de/programme/fis/
Contact:
Agentur für Innovation in der Cybersicherheit GmbH
Große Steinstraße 19
06108 Halle (Saale)
Michael Lindner
Press Officer
Tel.: +49 151 44150 645
Email: presse@cyberagentur.de
Background: Cyberagentur
The Agentur für Innovation in der Cybersicherheit GmbH (Cyberagentur) was founded in 2020 as a wholly owned subsidiary of the German Federal Government under the joint leadership of the German Federal Ministry of Defence and the German Federal Ministry of the Interior and Community with the aim of taking an application-strategy-oriented and cross-departmental view of internal and external security in the field of cybersecurity. Against this background, the work of the Cyberagentur is primarily aimed at the institutionalised implementation of highly innovative projects that carry a high risk of not achieving their objectives ( ) but which, at the same time, have a very high potential for disruption if successful.
The Cyberagentur is part of the Federal Republic of Germany's National Security Strategy.
The Cyberagentur is headed by Prof. Dr Christian Hummert as Scientific Director and Bettina Bubnys as Commercial Director.
Olivia Gräupner, Research Officer an Project Manager FIS
https://www.cyberagentur.de/en/press/ki-rechtssicher-gestalten/
https://www.cyberagentur.de/programme/fis/
Project launch for the ‘Forensics of Intelligent Systems’ (FIS) research programme with the three pr ...
Source: Cyberagentur
Copyright: Cyberagentur
Criteria of this press release:
Business and commerce, Journalists, Scientists and scholars, Students
Economics / business administration, Information technology, Law, Politics, Social studies
transregional, national
Cooperation agreements, Research projects
English
Project launch for the ‘Forensics of Intelligent Systems’ (FIS) research programme with the three pr ...
Source: Cyberagentur
Copyright: Cyberagentur
You can combine search terms with and, or and/or not, e.g. Philo not logy.
You can use brackets to separate combinations from each other, e.g. (Philo not logy) or (Psycho and logy).
Coherent groups of words will be located as complete phrases if you put them into quotation marks, e.g. “Federal Republic of Germany”.
You can also use the advanced search without entering search terms. It will then follow the criteria you have selected (e.g. country or subject area).
If you have not selected any criteria in a given category, the entire category will be searched (e.g. all subject areas or all countries).
