idw – Informationsdienst Wissenschaft
Nachrichten, Termine, Experten
Nachrichten, Termine, Experten
idw - Informationsdienst
Wissenschaft
Make machine intelligence less vulnerable
The Agentur für Innovation in der Cybersicherheit GmbH (Cyberagentur) has published a call for proposals on "Robust and Secure Machine Learning" on 6th March 2023. The goal is to increase the reliability and security against manipulation of various AI approaches. The multi-phase project aims to advance different innovation approaches during its five-year duration.
With "Robust and Secure Machine Learning" (RSML), Cyberagentur launches its largest research program on the security of artificial intelligence (AI) to date agency against the background of rapidly growing use of AI systems in all areas from society to industry.
In recent years, machine learning (ML) in various forms has become a powerful analysis and generation tool for ever larger and more complex data models due to constantly increasing levels of performance and versatility. One example is image recognition for autonomous driving. A fundamental problem: even inconspicuous changes to traffic signs can lead to malfunction of the driving algorithm. Research regarding guaranteed and verifiable behavior of AI models and systems is still in its infancy, exposing them to multiple vulnerabilities. Core problems include the great dependence on training data as well as the intransparency of trained models. As a result, unexpected inputs can lead to unforeseen and difficult-to-comprehend - and in some cases potentially catastrophic - results. Another example can be found in the current discussion surrounding AI chatbots. By means of manipulated commands ("prompt injection"), language models can demonstrably be misled into making unintended and potentially fateful statements.
It is against this background that the latest research project was developed by the scientists of Cyberagentur.
"The goal is to stimulate research and development of new, potentially groundbreaking approaches to increase the robustness and security of various AI approaches in the domains of internal and external security," says Dr Daniel Gille, project manager and head of "Security through AI and Security for AI" at Cyberagentur, explaining the approach of the research project call.
Five key research areas were identified for inclusion in the five-phase project competition: Automated data assurance, model verification, system integration, hybrid models of neural and symbolic components, and end-to-end verification. "We want to find answers to some of the big, unsolved questions in the often underrepresented field of security for AI," continues Dr Gille. "These are, for example: How and to what extent can security and robustness properties be proven for specific application domains and systems? What are the limits of provability and how can systems with desirable security properties be prototyped?"
"Especially in the context of internal and external security as well as critical infrastructures, the provability of security and robustness of neural AI systems against targeted attacks and manipulative inputs is of crucial importance," emphasises project manager Gille. Taking into account the current geopolitical situation, Cyberagentur wants to stimulate effective research with governmental and non-governmental actors who are technically capable and scientifically cutting-edge. The systems and infrastructures in focus are expected to directly impact the livelihoods of a potentially large number of citizens in the future. Maximum safety in the use of machine learning is therefore essential in these domains. "This cannot be guaranteed at the moment," Dr Gille continues. "Our planned commissioning is intended to help create and expand the scientific basis for this maximum cyber security of AI systems to be strived for."
During the five years of the program, basic research is to be conducted, followed by the development of prototypes. In order to realise innovative approaches across the five research areas for a broad spectrum of possible future applications, Cyberagentur plans to commission several bidders or bidding consortia in parallel. Within the framework of the envisaged "competition of ideas", results are to be evaluated at the end of each phase and the field of participants will be gradually reduced. The single most convincing approach will remain in the competition until the last phase is concluded. There, the participant will have the opportunity to test and evaluate developed artefacts in a realistic test environment. Bids can be submitted by all academic and non-academic research institutions, industrial companies as well as start-ups.
The call for proposals was published in the Supplement to the Official Journal of the European Union with the award number 2023/S 070-206022: https://www.evergabe-online.de/tenderdetails.html?id=505660.
Contact
Michael Lindner
Press Officer of the Cyberagentur
Phone: +49 151 44150 645
E-mail: presse@cyberagentur.de
Background: Cyberagentur
The Agentur für Innovation in der Cybersicherheit GmbH (Cyberagentur) was founded in 2020 as a fully in-house company of the Federal Government under the joint leadership of the Federal Ministry of Defence and the Federal Ministry of the Interior and for Home Affairs by the Federal Government with the goal of taking an application-strategy-related and interdepartmental view of internal and external security in the field of cybersecurity. Against this backdrop, the work of the Cyberagentur is primarily aimed at the institutionalised implementation of highly innovative projects that are associated with a high risk with regard to the achievement of objectives, but at the same time can have a very high disruptive potential if they are successful.
The Cyberagentur is headed by Prof. Dr. Christian Hummert as Research Director and Managing Director and Daniel Mayer as Commercial Director.
Dr. Daniel Gille
https://www.cyberagentur.de/bevor-maschinen-ueberlistet-werden/
https://www.evergabe-online.de/tenderdetails.html?id=505660
Dr. Daniel Gille, Head of Security by AI and Security for AI.
Andreas Stedtler
Cyberagentur
Criteria of this press release:
Business and commerce, Journalists, Scientists and scholars
Information technology, Mathematics, Physics / astronomy
transregional, national
Contests / awards, Research projects
English
You can combine search terms with and, or and/or not, e.g. Philo not logy.
You can use brackets to separate combinations from each other, e.g. (Philo not logy) or (Psycho and logy).
Coherent groups of words will be located as complete phrases if you put them into quotation marks, e.g. “Federal Republic of Germany”.
You can also use the advanced search without entering search terms. It will then follow the criteria you have selected (e.g. country or subject area).
If you have not selected any criteria in a given category, the entire category will be searched (e.g. all subject areas or all countries).
