idw – Informationsdienst Wissenschaft
Nachrichten, Termine, Experten
Nachrichten, Termine, Experten
An interdisciplinary team led by ethicist Professor Karoline Reinhardt and computational linguist Professor Annette Hautli-Janisz from the University of Passau is investigating a phenomenon in AI hallucinations that has received little attention to date: when large language models produce moral fallacies that deviate significantly from human values.
More and more people are using large language models (LLMs) such as GPT, PaLM or Llama – not just for simple information queries, but also for advice on practical matters. It is not uncommon for so-called hallucinations to occur when using such systems. This means that the LLMs provide answers that, whilst appearing realistic, are factually incorrect or misleading.
An interdisciplinary team led by ethicist Professor Karoline Reinhardt and computational linguist Professor Annette Hautli-Janisz is conducting research into a previously little-noticed type of such fabricated AI responses as part of the DFG project ‘Moral Hallucinations in Large Language Models’. The project focuses on moral hallucinations that can arise when people use LLMs for ethically relevant questions. For example, when they ask the machine whether it is always a moral duty to keep a promise.
“Moral hallucinations thus go beyond conventional AI hallucinations. They involve not only erroneous or unreliable information, but also distortions in moral argumentation structures,” explains Professor Reinhardt, holder of the Chair of Applied Ethics at the University of Passau. “And this often occurs in situations where people are vulnerable, seeking advice, or at a loss.”
The structure and consequences of moral AI hallucinations
This interdisciplinary research project at the University of Passau combines conceptual and empirical methods from applied AI ethics and computational linguistics with the aim of theoretically analysing, ethically evaluating and automatically identifying moral hallucinations generated by large language models (LLMs). To this end, the research team is investigating questions concerning both the structure of moral hallucinations and their ethical consequences. For example: What are the key characteristics of moral hallucinations? What ethical consequences arise if moral claims made by LLMs are not merely biased but are, in fact, hallucinations? What are the argumentative structures of moral hallucinations, and how do they relate to moral judgement in philosophical theories? What are the ethical consequences of using LLMs as moral advisors? How could LLMs be designed to flag moral hallucinations?
“There is an urgent need to expand existing benchmarks and evaluate models in greater detail, particularly in the field of moral reasoning,” says Professor Hautli-Janisz, holder of the Junior Professorship in Computational Rhetoric and Natural Language Processing. She adds that this is also an area that has received little attention in computational linguistics to date.
The project is funded by the German Research Foundation (DFG) for a period of three years as part of the DFG Priority Programme “Robust Assessment & Safe Applicability of Language Modelling: Foundations for a New Field of Language Science & Technology (LaSTing)” (SPP 2556). This is a funding programme launched in 2026 that brings together interdisciplinary research at the interface of linguistics, computer science, AI and philosophy, amongst other fields.
This text was machine-translated from German.
Professor Karoline Reinhardt
Professorship of Applied Ethics, University of Passau
E-Mail: Karoline.Reinhardt@uni-passau.de
Professor Annette Hautli-Janisz
Junior Professorship of Computational Rhetoric and Natural Language Processing, University of Passau
E-Mail: Annette.Hautli-Janisz@uni-passau.de
Symbolic image: A seamless transition between a human hand and a machine.
Source: Adobe Stock 1877298676
Copyright: Adobe Stock
Criteria of this press release:
Business and commerce, Journalists, Scientists and scholars, Students, Teachers and pupils, all interested persons
Information technology, Language / literature, Philosophy / ethics
transregional, national
Research projects
English
Symbolic image: A seamless transition between a human hand and a machine.
Source: Adobe Stock 1877298676
Copyright: Adobe Stock
You can combine search terms with and, or and/or not, e.g. Philo not logy.
You can use brackets to separate combinations from each other, e.g. (Philo not logy) or (Psycho and logy).
Coherent groups of words will be located as complete phrases if you put them into quotation marks, e.g. “Federal Republic of Germany”.
You can also use the advanced search without entering search terms. It will then follow the criteria you have selected (e.g. country or subject area).
If you have not selected any criteria in a given category, the entire category will be searched (e.g. all subject areas or all countries).
