idw – Informationsdienst Wissenschaft
Nachrichten, Termine, Experten
Nachrichten, Termine, Experten
Liberate AI, an interdisciplinary project uniting researchers from the medical domain, computer science, and trustworthy AI, aims to develop an AI model capable of supporting doctors in the treatment of ischemic stroke. Serving as a digital assistance system, the AI model is intended to predict the long-term outcome of patients after mechanical thrombectomy as well as potential complications. Using a technology called Swarm Learning, the AI model will be trained in a privacy-preserving fashion on medical data residing at different sites across Germany. The project also addresses the AI’s explainability as well as its ability to make differentiated predictions for patient subgroups.
Ischemic stroke occurs when blood clots become lodged in brain vessels, obstructing blood flow and, hence, oxygen supply. One possible treatment in this situation is mechanical thrombectomy, a minimally invasive procedure that uses a special catheter to remove vessel blockage. If mechanical thrombectomy is the most promising option for any given patient, however, depends on a number of case-specific factors. To assist doctors in making this time-critical decision, the researchers in Liberate AI seek to train an AI model on medical data stored in the German Stroke Register as well as associated MRI and CT scans held at various hospitals across Germany. To achieve this end, they leverage Swarm Learning, an AI technology developed by DZNE in cooperation with Hewlett Packard Enterprise. Swarm Learning effectively allows the AI to learn in a decentralized fashion, traveling to all the data repositories in the network to collect knowledge without the data itself leaving the sites where it is stored.
Toward a glass box: Explainability is key
While Swarm Learning is at the heart of Liberate AI, there are further technological challenges that go beyond the actual training of the AI model. The first of these challenges concerns explainability as one of the key characteristics that the AI model needs to exhibit. In contrast to deep-learning-based applications, which tend to operate on a black-box basis, the AI model developed in Liberate AI will have to make its reasoning transparent to the medical doctor using it. As CISPA-researcher Professor Dr. Jilles Vreeken, an expert on trustworthy information processing, explains: “We want to develop a glass-box AI that can predict as well as a black-box one. Because if you are a medical doctor and the AI says ‘yes’ or ‘no’, your first question is ‘Why should I trust you?’. This means we need to use explainable AI, which is the branch of AI research in which we develop AI models where we can understand based on which evidence they are saying what they are saying. This is the form of AI that can really support experts, because medical doctors will be able to tell whether the AI’s prediction is based on accidental evidence or on actual biomarkers.” While Vreeken and his research group aim at designing a transparent AI model, explainability poses specific technological challenges in the context of Swarm Learning. “We have to keep in mind that while we can develop this glass-box AI, it will still need to be able to learn in a swarm learning environment and to predict as reliably as a black-box one. It is not trivial to make that happen”, he says. In the project, the researchers will have to strike the balance between the AI model’s degree of transparency and its ability to be swarmed successfully.
In search of subpopulations and causal conclusions
The second major challenge addressed by the CISPA researchers concerns the identification of those patient populations that respond positively or negatively to mechanical thrombectomy in terms of quality of life over time. Ideally, the AI model will be able to automatically identify these statistical subgroups based on certain patterns that it extracts from the accumulated medical data is has digested. “The question is, can we develop a transparent box AI that can find conditions under which people have exceptional survival behavior? For example, this could depend on the size of the blood clot, high or low blood pressure, genetic factors, or the intake of blood thinners – you can think of certain conditions that will select some patients but not all of them”, Vreeken explains. These subgroups, he points out, can still be identified even if the training of an explainable glass-box AI should prove impossible in the Swarm Learning environment. “The beauty of our transparent box AI is that we can use it on top of a black-box AI, namely we can ask the question: For which people does the black-box AI tend to predict very well? This means that if we end up using a black-box AI, because it’s more accurate than any transparent model we can develop, we are still able to determine the subgroups for which we should or should not ask its opinion.”
Liberate AI: Combining domain expertise with machine learning
Ultimately, what the CISPA researchers would like to design is a transparent AI system capable of giving causal guarantees for its predictions. So that if, for instance, it predicted that high blood pressure will minimize the efficacy of the treatment, it could also give the reasons for this. “This is very difficult to do”, Vreeken says, “because you need a randomized control trial to determine if high blood pressure is the only factor or a confounder, something that seems to be relevant but that isn’t. So, the ultimate AI that we would like to develop is a glass-box AI capable of saying: Based on all available stroke data, there’s a clear difference between otherwise similar patients that cannot be explained in any other way except for blood pressure.”
Even if the tripartite challenge of offering explainability, subgroup identification, and causal guarantees should prove too ambitious in the end, Vreeken is certain that Liberate AI will be making a significant contribution to the applicability of AI in healthcare. The interdisciplinarity of the project team in particular opens up new possibilities for the treatment of acute stroke, as he highlights: “If you ask a domain expert what they want, they want a better machine X, but maybe they need something else that they don’t know is possible. The opposite problem is that computer scientists often develop new machines where the domain expert might say, this solves a problem that we do not have. I am very happy that in this project we have an excellent constellation of people with computer science expertise, people with pure medical domain expertise, and people in the middle. In Liberate AI, we will not be developing a machine that nobody is waiting for, but rather the machine people don't even know they need.”
This research project is supported by the Helmholtz Initiative and Networking Fund.
About Liberate AI
Liberate AI is a joint research project between researchers at Deutsches Zentrum für Neurodegenerative Erkrankungen (DZNE), the Department of Vascular Neurology at the University Hospital Bonn (UKB), and CISPA Helmholtz Center for Information Security. Liberate AI aims to develop a computer model based on artificial intelligence (AI) to aid doctors in the treatment of acute stroke patients, combining central registry with local brain imaging data. Serving as a digital assistance system, the model is intended to predict the long-term outcome of stroke patients after a minimally invasive treatment (mechanical thrombectomy) and potential complications, thereby helping doctors decide on the best possible therapy. Led by Prof. Dr. Joachim Schultze at DZNE, Liberate AI is funded by the Helmholtz Association with 250,000 euros.
About CISPA
The CISPA Helmholtz Center for Information Security is a national Big Science institution within the Helmholtz Association. It explores information security in all its facets in order to comprehensively and holistically address the pressing major challenges of cybersecurity and trustworthy artificial intelligence that our society faces in the digital age. CISPA holds a global leadership position in the field of cybersecurity, combining cutting-edge and often disruptive foundational research with innovative applied research, technology transfer, and societal discourse. Thematically, it aims to cover the entire spectrum from theory to empirical research. It is internationally recognized as a training ground for the next generation of cybersecurity experts and scientific leaders in the field.
Prof. Dr. Jilles Vreeken
CISPA Helmholtz Center for Information Security
Stuhlsatzenhaus 5
66123 Saarbrücken, Germany
vreeken@cispa.de
Towards a glass box AI: AI explainability in stroke treatment
Copyright: CISPA
Criteria of this press release:
Journalists, Scientists and scholars
Information technology, Medicine
transregional, national
Research projects
English
Towards a glass box AI: AI explainability in stroke treatment
Copyright: CISPA
You can combine search terms with and, or and/or not, e.g. Philo not logy.
You can use brackets to separate combinations from each other, e.g. (Philo not logy) or (Psycho and logy).
Coherent groups of words will be located as complete phrases if you put them into quotation marks, e.g. “Federal Republic of Germany”.
You can also use the advanced search without entering search terms. It will then follow the criteria you have selected (e.g. country or subject area).
If you have not selected any criteria in a given category, the entire category will be searched (e.g. all subject areas or all countries).
