idw – Informationsdienst Wissenschaft
Nachrichten, Termine, Experten
Nachrichten, Termine, Experten
idw - Informationsdienst
Wissenschaft
The enormous computing resources needed to train neural networks for artificial intelligence (AI) result in massive power consumption. Researchers at the Technical University of Munich (TUM) have developed a method that is 100 times faster and therefore much more energy efficient. Instead of taking an iterative approach, the parameters are computed directly based on probabilities. The results so far are comparable in quality to existing iterative methods.
AI applications such as large language models (LLMs) have become an integral part of our everyday lives. The required computing, storage and transmission capacities are provided by data centers that consume vast amounts of energy. In Germany alone this amounted to around 16 billion kWh in 2020, or around 1% of the country’s total energy consumption. For 2025, this figure is expected to increase to 22 billion kWh.
New method is 100 times faster with comparable accuracy
The arrival of more complex AI applications in the coming years will substantially increase the demands on data center capacity. These applications will use up huge amounts of energy for the training of neural networks. To counteract this trend, researchers have developed a training method that is 100 times faster while achieving accuracy comparable to existing procedures. This will significantly reduce the energy consumption for training.
The functioning of neural networks, which are used in AI for such tasks as image recognition or language processing, is inspired by way the human brain works. These networks consist of interconnected nodes called artificial neurons. The input signals are weighted with certain parameters and then summed up. If a defined threshold is exceeded, the signal is passed on to the next node. To train the network, the initial selection of parameter values is usually randomized, for example using a normal distribution. The values are then incrementally adjusted to gradually improve the network predictions. Because of the many iterations required, this training is extremely demanding and consumes a lot of electricity.
Parameters selected according to probabilities
Felix Dietrich, a professor of Physics-enhanced Machine Learning, and his team have developed a new method. Instead of iteratively determining the parameters between the nodes, their approach uses probabilities. Their probabilistic method is based on the targeted use of values at critical locations in the training data where large and rapid changes in values are taking place. The objective of the current study is to use this approach to acquire energy-conserving dynamic systems from the data. Such systems change over the course of time in accordance with certain rules and are found in climate models and in financial markets, for example.
“Our method makes it possible to determine the required parameters with minimal computing power. This can make the training of neural networks much faster and, as a result, more energy efficient,” says Felix Dietrich. “In addition, we have seen that the accuracy of the new method is comparable to that of iteratively trained networks.”
Prof. Dr. Felix Dietrich
Physics-enhanced Machine Learning
Technical University of Munich
felix.dietrich@tum.de
www.tum.de
Rahma, Atamert, Chinmay Datar, and Felix Dietrich, “Training Hamiltonian Neural Networks without Backpropagation”, 2024. Machine Learning and the Physical Sciences
Workshop at the 38th conference on Neural Information Processing Systems (NeurIPS)
https://neurips.cc/virtual/2024/99994
https://www.tum.de/en/news-and-events/all-news/press-releases/details/new-method...
Felix Dietrich, Professor of Physics-enhanced Machine Learning
Andreas Heddergott / TUM
Andreas Heddergott / TUM; Free for use in reporting on TUM, with the copyright noted
Criteria of this press release:
Journalists
Information technology
transregional, national
Research results, Scientific Publications
English
You can combine search terms with and, or and/or not, e.g. Philo not logy.
You can use brackets to separate combinations from each other, e.g. (Philo not logy) or (Psycho and logy).
Coherent groups of words will be located as complete phrases if you put them into quotation marks, e.g. “Federal Republic of Germany”.
You can also use the advanced search without entering search terms. It will then follow the criteria you have selected (e.g. country or subject area).
If you have not selected any criteria in a given category, the entire category will be searched (e.g. all subject areas or all countries).
