idw – Informationsdienst Wissenschaft
Nachrichten, Termine, Experten
Nachrichten, Termine, Experten
To deploy humanoid robots in industrial applications, they must meet application-relevant criteria. Fraunhofer IPA has developed a benchmark for this purpose. This allows manufacturers and end users to have humanoid robots analyzed by an independent third party for energy efficiency, cleanroom compatibility, data security, and more.
The Fraunhofer Institute for Manufacturing Engineering and Automation IPA has developed a comprehensive benchmark for the standardized analysis of humanoid robots. For the first time, this allows manufacturers and end users to have the actual capabilities, safety, and operational suitability of these robots objectively evaluated by an independent third party. The modular benchmark comprises six application-relevant criteria and is based on internationally recognized industry standards.
From media presence to realistic assessment
Humanoid robots are omnipresent in the media and fascinate with their human-like appearance. Yet there is a vast gap between spectacular presentations and actual capabilities. “For end users and manufacturers alike, it is essential to look behind the facade sometimes constructed by marketing agencies,” explains Simon Schmidt, Senior Manager of the Business Unit Automated Systems at Fraunhofer IPA. “The market is too volatile and opaque to allow for a well-founded assessment and reliable evaluation of humanoids for specific applications.”
What is the benchmark?
The Benchmark is a standardized service in which research teams at Fraunhofer IPA guide humanoid robots through various challenges and scientifically evaluate the results. The foundations for this were laid thanks to funding from the Baden-Württemberg Ministry of Economic Affairs, Labour, and Tourism as part of the AI Innovation Center “Learning Systems and Cognitive Robotics.”
The modular structure enables manufacturers, end users, and software providers to specifically test the areas relevant to their application. Where possible, the benchmarking is based on established industry standards that have been internationally recognized for decades – for example, ISO 14644 for cleanroom suitability or ISO 10218 and ISO TS 15066 for functional safety.
The benchmark is divided into six key areas:
1. Technologies and basic capabilities: Examination of the installed sensors, AI models, and gripper types, as well as tests of walking speed, gripping forces, and manageable loads. Objective measurements are recorded using a 3D tracking system and force sensors.
2. Complex capabilities: Evaluation of practical generic tasks such as stair climbing, obstacle navigation, movement and force precision, and reaction speed. The tests are deliberately designed to be challenging to ensure comparability with future robot generations.
3. Cleanroom suitability: Evaluation of particle emission according to ISO 14644-14, outgassing behavior, and cleanability, all critical for applications for example in the semiconductor, pharmaceutical, or food industries.
4. Functional safety: Central to human-robot collaboration. Tests include stability on various surfaces, force limitation during collisions, obstacle detection, and system behavior during failures. Collision tests are conducted using the same force sensors as those used for collaborative industrial robots.
5. Cybersecurity: Four modules assess vulnerability management, secure lifecycle, network security, and penetration resistance. These are important factors given the increasing regulatory requirements.
6. Energy efficiency: Measurement of battery life and power consumption in various scenarios (standing, walking, walking uphill, and walking with a load). The results enable realistic deployment planning and optimization of charging cycles.
Using the Unitree G1 as an example, Fraunhofer IPA applied the benchmark comprehensively for the first time. The technical basis was the Unitree G1 EDU-4 with Dex3-1 3-finger hands delivered in May 2025 with firmware version 1.04.
While the robot demonstrates good self-stabilization and could be suitable for ISO Class 5 cleanrooms, significant limitations also became apparent. In the event of collisions, forces exceeding 500 newtons can occur, what is far above the pain thresholds permitted by the standard. In addition, the researchers identified a critical Bluetooth security vulnerability in the software at that time that allowed attackers to take complete remote control. This issue has since been resolved. In terms of energy efficiency, the maximum operating times on a single battery charge were 2 hours and 49 minutes while standing still and 1 hour and 49 minutes in a typical scenario involving both standing and walking.
Relevance of the Benchmark for Companies
“Users can interpret the results directly and thus find the right humanoid for the right application,” emphasizes Werner Kraus, head of the research division “Automation and robotics” at Fraunhofer IPA. The benchmark makes humanoids comparable not only with each other but also with proven automation components. This is particularly important because:
- demographic change is forcing the use of automation in areas previously handled manually
- major investment decisions require well-founded, objective evaluation criteria
- safety standards for humanoids are not expected until 2028 (ISO 25785-1)
- regulatory requirements for cybersecurity are increasing
- sensitive production environments require reliable data to prevent contamination
The benchmark provides transparency in an opaque market and enables companies to develop realistic expectations and minimize risks.
Fraunhofer IPA plans to test additional humanoids and establish a comparative database. Manufacturers and users can now commission individual benchmark modules up to comprehensive evaluations and benefit from the existing infrastructure and expertise.
Dr.-Ing. Werner Kraus | Phone +49 711 970-1049 | werner.kraus@ipa.fraunhofer.de
Simon Schmidt | Mobile Phone +49 172 5418428 | simon.schmidt@ipa.fraunhofer.de
The tests and benchmarks are divided into six categories.
Copyright: Fraunhofer IPA
The ability to self-stabilize is examined through movements with different paths on a ramp.
Source: Rainer Bez
Copyright: Fraunhofer IPA
Press release as PDF
Criteria of this press release:
Business and commerce, Journalists, Scientists and scholars
Mechanical engineering
transregional, national
Transfer of Science or Research
English
The tests and benchmarks are divided into six categories.
Copyright: Fraunhofer IPA
The ability to self-stabilize is examined through movements with different paths on a ramp.
Source: Rainer Bez
Copyright: Fraunhofer IPA
You can combine search terms with and, or and/or not, e.g. Philo not logy.
You can use brackets to separate combinations from each other, e.g. (Philo not logy) or (Psycho and logy).
Coherent groups of words will be located as complete phrases if you put them into quotation marks, e.g. “Federal Republic of Germany”.
You can also use the advanced search without entering search terms. It will then follow the criteria you have selected (e.g. country or subject area).
If you have not selected any criteria in a given category, the entire category will be searched (e.g. all subject areas or all countries).
