idw – Informationsdienst Wissenschaft
Nachrichten, Termine, Experten
Nachrichten, Termine, Experten
‘X-hacking’ is the term used by DFKI researchers to describe a risk in the field of explainable artificial intelligence (XAI) that has received little attention to date. At this year's International Conference on Machine Learning (ICML), one of the world's most important conferences for machine learning, the team from the Data Science and its Applications research department is presenting a systematic analysis of this phenomenon for the first time - and appealing for a critically considered use of AutoML tools.
‘At a time when AI explains decisions but does not always understand them, we as scientists must take responsibility for the depth of these explanations - and for their limitations’, says Prof. Sebastian Vollmer, head of research department Data Science and its Applications at DFKI in context of the ICML 2025. He pleads for reflected use of AutoML in research and practice.
What happens when AI systems make correct predictions but give completely different reasons for how they arrive at this result? Can users then not simply choose the explanation that best fits their desired narrative? The DFKI team led by Prof Sebastian Vollmer (Rahul Sharma, Sumantrak Mukherjee, Andrea Šipka, Eyke Hüllermeier, Sergey Redyuk and David Antony Selby) investigated precisely this problem and identified a structural risk to the trustworthiness of AI with the term X-hacking.
X-hacking: When AutoML plausibly deceives
The term X-hacking, based on the p-hacking known from statistics, describes two central mechanisms:
Cherry-Picking: from a large number of similarly good models, the one whose explanation best supports the desired result is specifically selected.
Directed search: AutoML systems not only optimise the prediction performance, but also specifically find models with certain explanation patterns - an often underestimated risk.
The problem is that feature importance - i.e. the weighting of input features - can differ drastically, even if the models deliver almost identically good results. This is particularly sensitive in fields of application such as medical research or social science, where explainable models often form the basis for critical decisions.
‘The explainability of a model can become an illusion, especially when there are many plausible but contradictory models to choose from.’, says David Antony Selby, researcher at Data Science and its Applications.
What is behind AutoML - the core of the problem?
AutoML (Automated Machine Learning) stands for automated processes for the development, selection and optimisation of ML models. Software tools take over many tasks that were previously reserved for experienced ML engineers, such as the selection of suitable model architectures, preprocessing steps and parameter tuning.
Especially in data-intensive fields such as medicine, industry or social research, AutoML tools promise faster development, lower access barriers and reproducible results. However, it is precisely this automation that makes it difficult to trace the origins of modelling decisions - a critical aspect of explainable AI. The best-known AutoML frameworks include auto-sklearn, Google Cloud AutoML, H2O.ai und Microsoft Azure AutoML.
Solution approach: Honest explainability through transparency
The DFKI team is deliberately not proposing any technical control mechanisms, but rather a scientifically reflected practice based on transparency and methodological diligence. The following recommendations take centre stage:
1. explanation histograms:
show the distribution of model explanations across all valid models and help to recognise outliers immediately.
2. complete pipeline documentation:
not only should the result be disclosed, but also the entire search space of models, data pre-processing and evaluation metrics.
3. interdisciplinary training:
Specialist disciplines using AutoML should be aware of the methodological risks and not just trust the software.
‘The goal is a scientific culture that focuses not only on accuracy, but also on honesty in explainability’, says Vollmer.
Trustworthy AI as a DFKI focus
The ICML 2025 study emphasises DFKI's research approach of making artificial intelligence not only powerful, but also transparent and socially trustworthy. In the context of the strategic focus ‘Trustworthy AI’, this work is an example of how scientific excellence and methodological responsibility can be combined.
Prof. Dr. Sebastian Vollmer
Head research department Data Science and its Applications, DFKI
Sebastian.Vollmer@dfki.de
Phone: +49 631 20575 7601
back arrowGo to ICML 2025 Conference homepage
X-Hacking: The Threat of Misguided AutoML
Authors: Rahul Sharma, Sumantrak Mukherjee, Andrea Sipka, Eyke Hüllermeier, Sebastian Josef Vollmer, Sergey Redyuk, David Antony Selby
https://openreview.net/forum?id=Bb0zKbPE0L
https://ICML-Poster X-Hacking: The Threat of Misguided AutoML
https://icml.cc/virtual/2025/poster/46106
At ICML 2025: New study on ‘X-Hacking’ shows risks of automated model selection
Source: DFKI
Copyright: DFKI
Criteria of this press release:
Journalists
Information technology, Mathematics, Philosophy / ethics
transregional, national
Research results, Scientific conferences
English
At ICML 2025: New study on ‘X-Hacking’ shows risks of automated model selection
Source: DFKI
Copyright: DFKI
You can combine search terms with and, or and/or not, e.g. Philo not logy.
You can use brackets to separate combinations from each other, e.g. (Philo not logy) or (Psycho and logy).
Coherent groups of words will be located as complete phrases if you put them into quotation marks, e.g. “Federal Republic of Germany”.
You can also use the advanced search without entering search terms. It will then follow the criteria you have selected (e.g. country or subject area).
If you have not selected any criteria in a given category, the entire category will be searched (e.g. all subject areas or all countries).
