idw – Informationsdienst Wissenschaft
Nachrichten, Termine, Experten
Nachrichten, Termine, Experten
idw - Informationsdienst
Wissenschaft
Data constantly changes. For instance, the name of the current German Chancellor can change faster than expected. Machine learning models must adapt to new knowledge and continually improve their ability to solve different tasks. To date, this has typically meant they need to be retrained regularly. This is not only expensive but also demands enormous computational power. CISPA-Faculty Dr. Sebastian Stich aims to address this with his research project "Collective Minds," which seeks to develop new algorithms for more sustainable and efficient ML. According to Stich, the key is better collaboration. The European Research Council will fund his project with approximately 2 million euros.
Lifelong learning is a goal not just for humans. Intelligent systems also need to continually learn and adapt. The problem: When machine learning models are repeatedly confronted with new data or changing tasks, they, like humans, tend to forget what they previously learned. However, the extent of this is so severe that experts refer to it as "catastrophic forgetting." As a result, it is still common in many fields to periodically retrain models from scratch. This retraining is extremely inefficient and unsustainable, comparable to humans needing to relearn how to walk just to keep up with the latest political developments.
Collaborative Learning: How Small Models Can Achieve Big Results Together
"The high costs of retraining and the immense computational demands make machine learning models a luxury that often only large industrial companies can afford," explains CISPA researcher Sebastian Stich. Yet, fields like medicine could significantly benefit from their capabilities. Approaches to address this issue have been around for a while. One way to save resources is to move away from training one large and complex model toward training several smaller models that share their knowledge rather than their training data. "This is referred to as federated learning, a type of collaborative learning. Existing methods focus on using distributed training data to ultimately train a large model. We are pursuing a different approach: we want small, independent models to collaborate effectively," Stich says.
Better Knowledge Sharing, Continuous Learning, and Meaningful Forgetting
His plan includes three core objectives: "We aim to use improved training algorithms to make models better at adapting to new data and forgetting outdated knowledge. Additionally, we will work on enabling models trained on different devices and with various data sources to share their knowledge effectively. Each of these models develops a type of specialized expertise based on the data it was trained on and can be used for specific tasks. Lastly, we want to enable these small expert models to collaboratively tackle complex tasks," Stich explains.
The result would be a more sustainable, adaptable, and equitable form of machine learning that benefits not only large corporations but also smaller players. "Our research could also be extremely useful in medicine. Patient data is both scarce and sensitive. If local hospitals can work with small models, it helps protect data, conserve resources, and still allow models to improve collaboratively, ultimately providing better support, for instance, in cancer diagnostics," Stich says.
The EU Supports Innovative and Independent Research
In 2024, a total of 328 researchers from 25 countries across Europe will receive a Consolidator Grant. Sixteen of these prestigious grants are awarded in the field of computer science and informatics. The EU, through the ERC Grants, supports outstanding researchers in establishing an independent research team, enabling them to implement their most promising research ideas. Receiving an ERC Consolidator Grant is an honor for Stich as well as for his peers. "The fact that the EU believes in my research project means a lot to me. This support is not only recognition of our previous work but also an incentive to continue developing innovative solutions to the challenges of machine learning. I am convinced that our research can make a significant contribution to a more sustainable and equitable technological future," Stich says.
About the ERC
The ERC, set up by the European Union in 2007, is the premier European funding organisation for excellent frontier research. It funds creative researchers of any nationality and age, to run projects based across Europe. The ERC offers four core grant schemes: Starting Grants, Consolidator Grants, Advanced Grants and Synergy Grants. With its additional Proof of Concept Grant scheme, the ERC helps grantees to bridge the gap between their pioneering research and early phases of its commercialisation. The ERC is led by an independent governing body, the Scientific Council. Since November 2021, Maria Leptin is the President of the ERC. The overall ERC budget from 2021 to 2027 is more than €16 billion, as part of the Horizon Europe programme, under the responsibility of European Commissioner for Startups, Research and Innovation, Ekaterina Zaharieva.
Sebastian Stich stich@cispa.de
CISPA-Faculty Dr. Sebastian Stich receives ERC Consolidator Grant for his research on more sustainab ...
Criteria of this press release:
Journalists, Scientists and scholars
Information technology
transregional, national
Contests / awards, Research projects
English
You can combine search terms with and, or and/or not, e.g. Philo not logy.
You can use brackets to separate combinations from each other, e.g. (Philo not logy) or (Psycho and logy).
Coherent groups of words will be located as complete phrases if you put them into quotation marks, e.g. “Federal Republic of Germany”.
You can also use the advanced search without entering search terms. It will then follow the criteria you have selected (e.g. country or subject area).
If you have not selected any criteria in a given category, the entire category will be searched (e.g. all subject areas or all countries).
