idw – Informationsdienst Wissenschaft
Nachrichten, Termine, Experten
Nachrichten, Termine, Experten
- Using an avatar reduces stress for patients during an autonomous ultrasound examination.
- The avatar answers questions in any language during an ultrasound examination.
- Users see the avatar through VR glasses.
Patients have more confidence in autonomous robotic ultrasound systems when an avatar guides them through the process. This was discovered by Prof. Nassir Navab from the Technical University of Munich (TUM). The virtual agent explains what it is doing, answers questions and can speak any language. Such systems are intended especially for use in regions where there is a shortage of doctors.
A large screen, virtual reality glasses, a robotic arm with an ultrasound head and a powerful computer: that is the equipment needed by TUM researcher Tianyu Song from the Chair of Informatics Applications in Medicine for the autonomous examination of the aorta, carotid artery or forearm arteries. To help overcome possible doubts about the autonomous technical system, researchers have now created a virtual environment in which an avatar guides patients through the examination procedure. After putting on VR glasses, patients see an avatar that leads the conversation and answers questions. “This makes the whole process more human and friendly,” says Prof. Nassir Navab, the head of the research chair. “And this has been proven to reduce stress among users of autonomous systems.”
Virtual environment reduces patients’ stress levels
To find out more, the researchers compared the stress levels of 14 male and female patients of varying ages. Three of the four scenarios were more or less virtually supported. In one scenario, an avatar was used in a real environment, another in a virtual environment in which real elements are superimposed, and the third in a completely virtual environment. These were compared with an avatar-free, purely real variant. The researchers wired the test subjects with sensors for an electrocardiogram (ECG) to record heart rate variability. “The more this value drops during treatment, the higher the stress level of the person being treated,” explains Tianyu Song. The result: all three virtually supported scenarios proved significantly less stressful than the non-virtual treatment. When asked which of the three virtually supported scenarios they trusted the most and which felt best, the avatar in a real environment came out on top. “That’s why we’re now using it for demonstrations,” says Professor Navab, whose research is supported by the Bavarian Research Foundation as part of the ForNeRo research project (Research Network – Seamless and Ergonomic Integration of Robotics into Clinical Workflows).
Large language model masters accents
The main reason for the reduced stress levels of those being treated is the avatar, which usually has a female voice in the department’s demonstrations and walks the patient through the examination. It holds the ultrasound probe and guides it to the arm. It also talks to the patient. To make this possible, software converts the patient’s questions into text before a large language model finds suitable answers based on pre-formulated instructions, which are then converted back into spoken words. “An important trust-building factor is the fact that the avatar not only speaks different languages, but can even do so in regional accents,” says researcher Song. For example, the language model can speak with an Austrian accent or even German with an American accent. The avatar can also communicate non-verbally. It gestures, pauses briefly between sentences, and turns to face patients when they speak.
Further information
- Research Project ForNeRo: https://fornero.ed.tum.de/
Additional press materials:
Photos: https://mediatum.ub.tum.de/1796709
Prof. Nassir Navab
Chair for Computer Aided Medical Procedures & Augmented Reality
Technical University of Munich (TUM)
nassir.navab@tum.de
Tianyu Song , Felix Pabst, Ulrich Eck, Nassir Navab; Enhancing Patient Acceptance of Robotic Ultrasound through Conversational Virtual Agent and Immersive Visualizations; IEEE Transactions on Visualization and Computer Graphics, 5-2025; https://ieeexplore.ieee.org/document/10916942
Criteria of this press release:
Journalists, Scientists and scholars, all interested persons
Information technology, Medicine
transregional, national
Miscellaneous scientific news/publications, Research results
English
You can combine search terms with and, or and/or not, e.g. Philo not logy.
You can use brackets to separate combinations from each other, e.g. (Philo not logy) or (Psycho and logy).
Coherent groups of words will be located as complete phrases if you put them into quotation marks, e.g. “Federal Republic of Germany”.
You can also use the advanced search without entering search terms. It will then follow the criteria you have selected (e.g. country or subject area).
If you have not selected any criteria in a given category, the entire category will be searched (e.g. all subject areas or all countries).
