idw – Informationsdienst Wissenschaft
Nachrichten, Termine, Experten
Nachrichten, Termine, Experten
AI training under fire – Cyberagentur focuses on defence strategy
Phase 4 of the RSML programme kicks off with a focus on secure AI training processes
Following a successful evaluation of project phase 3, the Agentur für Innovation in der Cybersicherheit GmbH (Cyberagentur) is beginning phase 4 of its research programme "Robust and Secure Machine Learning" (RSML). Three remaining consortia are now working on securing the training process of AI systems against manipulation – a further step towards verifiably robust AI applications for critical infrastructures.
Phase 4 of the RSML research competition has started. Following the recent successful evaluation of phase 3, the three remaining contractors are moving on to the next round. The new project phase focuses on securing AI training ("training security"). At the end of phase 4, there will be another interim evaluation of the results as part of the competitive process to measure progress and, if necessary, further narrow down the field of participants. This keeps the aim firmly in sight: the development of demonstrably robust and secure AI systems for use in safety-critical areas such as energy, health and communications infrastructures.
The RSML programme was announced in 2023 and will run for five years. It will be carried out competitively in several phases and financed by the federal government with a total of 25 million euros. In terms of content, the programme is divided into five central research pillars:
• Data security – automated assurance of data quality and integrity
• Model verification – formal verification of AI models for correctness and robustness,
• System embedding – secure integration of AI into operational systems and processes,
• Hybrid models – combining symbolic and neural AI approaches for greater transparency,
• End-to-end verification – continuous verification of security guarantees throughout the entire life cycle.
This research programme is part of the German Federal Government's National Security Strategy and its results address the growing security requirements in the field of AI. As part of the RSML competition, the Cyberagentur is financing several solution approaches in parallel. Each phase concludes with an evaluation, and only the most convincing concepts advance to the next round. The most visionary solution will ultimately be tested in a realistic test environment to demonstrate its potential for Germany's internal and external security needs.
Phase 3 of the RSML programme, which ran from December 2024 to November 2025, focused on data security and was recently successfully completed. The results from this phase to date underscore the importance of a tamper-proof data basis for AI systems. "The data used for training is often incomplete, poorly labelled or even manipulated. This is the root cause of many subsequent security and reliability problems, for which we want to find solutions," emphasised Dr Daniel Gille, RSML project manager, during the previous project phase. Accordingly, the three consortia in phase 3 focused on innovative approaches to securing the data basis of AI models. Among other things, a modular toolkit with end-to-end evaluation processes was developed, providing new metrics and tools for the targeted development of reliable ML systems. A second approach explored hybrid AI-supported red/blue group agents that mutually test vulnerable systems and their security mechanisms to uncover vulnerabilities. Thirdly, a holistic verification framework was created with comprehensive threat modelling and a central "RSML Operations Centre" as an interface. These results form the basis for the next phase and have been incorporated into the planning for Phase 4.
With Phase 4, the RSML programme is now addressing the next critical building block: securing the training process for neural AI models. The aim is to ward off targeted attacks – such as data manipulation or poisoned training data – while AI systems are still learning, so that the models are robust and trustworthy from the outset. "Especially in critical infrastructures, the training process of an AI must be protected against manipulation from the outset," emphasises Dr Daniel Gille, Head of Artificial Intelligence and RSML Project Manager at the Cyberagentur. "Training security forms the basis for developing AI systems with verifiable robustness and reliability for safety-critical applications. If the learning process of an AI is already secure, we can make its use in areas such as energy supply, transport or communication much more trustworthy." The focus on training security in phase 4 sets the course for future AI applications to be not only based on secure data, but also trained securely.
Phase 4 now underpins the overarching importance of the RSML programme for AI security. Research into robust and secure machine learning is making an important contribution to ensuring that Germany has reliable and verifiable AI systems in an age of increasing AI use – especially in areas where malfunctioning could have serious consequences. The RSML programme demonstrates how innovative research funding and competition are opening up new ways to equip AI systems against current and future threats. With the completion of phase 4 and the upcoming interim evaluation, the overall project is one step closer to its aim: to ultimately produce an AI system that functions safely even under the most adverse conditions, thereby strengthening confidence in AI technologies in safety-critical applications.
Further information:
https://www.cyberagentur.de/programme/rsml/
Contact:
Agency for Innovation in Cybersecurity GmbH
Große Steinstraße 19
06108 Halle (Saale)
Michael Lindner
Press Officer
Tel.: +49 151 44150 645
Email: presse@cyberagentur.de
Background: Cyberagentur
The Agentur für Innovation in der Cybersicherheit GmbH (Cyberagentur) was founded in 2020 as a wholly owned subsidiary of the German Federal Government under the joint leadership of the German Federal Ministry of Defence and the German Federal Ministry of the Interior and Community with the aim of taking an application-strategy-oriented and cross-departmental view of internal and external security in the field of cybersecurity. Against this background, the work of the Cyberagentur is primarily aimed at the institutionalised implementation of highly innovative projects that carry a high risk in terms of achieving their objectives, but at the same time have a very high potential for disruption if successful.
The Cyberagentur is part of the Federal Republic of Germany's National Security Strategy.
The Cyberagentur is headed by Prof. Dr Christian Hummert as Scientific Director and Bettina Bubnys as Commercial Director.
Dr. Daniel Gille, Head of Artificial Intelligence
https://www.cyberagentur.de/en/press/ki-training-unter-beschuss-cyberagentur-set...
https://www.cyberagentur.de/en/programs/rsml/
Dr. Daniel Gille, Head of Artificial Intelligence at the Cyberagentur.
Source: Nancy Glor
Copyright: Cyberagentur
Criteria of this press release:
Business and commerce, Journalists, Scientists and scholars, Students
Economics / business administration, Electrical engineering, Information technology, Mathematics, Physics / astronomy
transregional, national
Research projects, Research results
English
Dr. Daniel Gille, Head of Artificial Intelligence at the Cyberagentur.
Source: Nancy Glor
Copyright: Cyberagentur
You can combine search terms with and, or and/or not, e.g. Philo not logy.
You can use brackets to separate combinations from each other, e.g. (Philo not logy) or (Psycho and logy).
Coherent groups of words will be located as complete phrases if you put them into quotation marks, e.g. “Federal Republic of Germany”.
You can also use the advanced search without entering search terms. It will then follow the criteria you have selected (e.g. country or subject area).
If you have not selected any criteria in a given category, the entire category will be searched (e.g. all subject areas or all countries).
