idw – Informationsdienst Wissenschaft
Nachrichten, Termine, Experten
Nachrichten, Termine, Experten
idw - Informationsdienst
Wissenschaft
Researchers at Graz University of Technology are using virtual reality and large language models to support people with autism spectrum disorder in training social skills. The system is intended to make treatment options more widely accessible.
An increasing number of people worldwide are affected by autism spectrum disorder (ASD); according to studies, one in 44 children is diagnosed with it. A central symptom is so-called “social blindness”, i.e. the inability to recognise emotions in others and to react appropriately to social situations. Suitable therapy is usually based on one-to-one or small group support, which is only available to a limited extent and is cost-intensive. Researchers at the Institute of Human-Centred Computing at Graz University of Technology (TU Graz) are using computer game technology to create an effective supplement that is inexpensive and available at any time. Initial studies show that this approach helps people with ASD to get through everyday life more safely.
Everyday situations without social consequences
The specially developed virtual environment Simville uses virtual reality, large language models (LLMs), speech recognition and speech generation to make social training location-independent and therefore more accessible for those affected. In this computer world, users train for realistic everyday situations, such as conversations with work colleagues or meeting people in a café. As this takes place in a controlled environment, users can act freely without having to fear social consequences. These training scenarios make them better prepared for similar interactions in everyday life.
“Our system is not meant to replace conventional therapies, but to complement and enhance them in a meaningful way,” says Christian Poglitsch from the Institute of Human-Centred Computing at TU Graz, who implemented the project as part of his doctoral thesis. The immersive but playful approach is of central importance to Simville. Tasks, storytelling and immediate feedback after acting out a scene motivate participants to practise regularly. In addition, the number of stimuli acting on the user can be controlled, so that beginners can start with a small number and increase this over time through their training and reduce it again if they become overwhelmed.
Language model conveys emotions
By integrating LLMs as well as speech recognition and generation, users can speak to the avatars in the game world like normal. What is said is converted into text by the speech recognition system, and a large language model creates a reaction tailored to the situation and the avatar responds accordingly in spoken language. The team used the model Gemini 12B from Google to create and play out the response. “What was fascinating was that the model was also able to convey a certain emotion. Depending on the context of what is being said, you can definitely hear the right undertone,” says Christian Poglitsch.
Test subjects feel safer
Initial studies show that training with Simville has positive effects. A study of 25 participants showed that after just a few sessions, many felt much more confident in social situations. Simville is now being incorporated into the international ETAP project led by Furtwangen University. The simulation interface is combined with extensive sensor technology in order to reduce or increase the intensity of the experience based on the user’s reaction. In addition, the Game Lab Graz at TU Graz would like to make Simville available as a demonstrator so that affected people can train with it themselves.
Christian POGLITSCH
Mag.rer.soc.oec. Dipl.-Ing. Bakk.rer.soc.oec. BSc
TU Graz | Institute of Human-Centred Computing
christian.poglitsch@tugraz.at
Large language models for autism: evaluating theory of mind tasks in a gamified environment https://doi.org/10.1038/s41598-025-18608-4
Re-enacting everyday situations is intended to prepare people for similar situations.
Copyright: HCC - TU Graz
Training on how to place an order in a restaurant.
Copyright: HCC - TU Graz
Criteria of this press release:
Journalists, Scientists and scholars, all interested persons
Information technology, Medicine, Psychology
transregional, national
Research results, Transfer of Science or Research
English
You can combine search terms with and, or and/or not, e.g. Philo not logy.
You can use brackets to separate combinations from each other, e.g. (Philo not logy) or (Psycho and logy).
Coherent groups of words will be located as complete phrases if you put them into quotation marks, e.g. “Federal Republic of Germany”.
You can also use the advanced search without entering search terms. It will then follow the criteria you have selected (e.g. country or subject area).
If you have not selected any criteria in a given category, the entire category will be searched (e.g. all subject areas or all countries).
