idw – Informationsdienst Wissenschaft

Nachrichten, Termine, Experten

Grafik: idw-Logo
Science Video Project
idw-Abo

idw-News App:

AppStore

Google Play Store



Instanz:
Teilen: 
07.06.2023 10:45

Powerful object recognition for diverse applications using AI and robotics

Dr. Karin Röhricht Presse- und Öffentlichkeitsarbeit
Fraunhofer-Institut für Produktionstechnik und Automatisierung IPA

    New algorithms for object recognition and environment perception have been developed at Fraunhofer IPA, enabling sophisticated automation solutions for the service and production sectors. They are a result of the “Knowledge4Retail” research project, in which twelve project partners have developed solutions for future-oriented retail.

    Making stationary retail more attractive to customers again with the help of digital solutions – this was the goal of the partners working together on the Knowledge4Retail (K4R) project from January 2020 to December 2022. The group presented the results at the final meeting this spring. The core of the project is a K4R open-source platform available on internet host GitHub, a version control system, which makes the results accessible to retailers. Retail companies can use the platform to connect analog and digital worlds as well as analyze processes more easily and generate purchasing behavior forecasts, for example. In the process, the project partners developed solutions for applications in logistics, store set-up, service robotics and a fully automated kiosk.

    Fast, flexible 3D image processing

    The IPA researchers worked on several development focuses as part of the project, during the course of which they expanded their expertise as well as the resulting range of 3D image processing-based products and services. One core aspect was the creation of what is known as semantic digital twinning of retail products. This involves using sensors to capture any product in a scanning station, for example. The product is then modeled three-dimensionally based on a wide range of information such as shape, color, and texture. The modeling process, in which Fraunhofer IPA collaborated with the Kaptura company, also involved machine learning methods, a subfield of AI (Artificial Intelligence).

    With the help of the technologies developed, it is possible to model wide product ranges quickly and in 3D with high resolution texture. During the modeling process, semantic information such as logos, nutritional value or quantitative information on products is also recognized and digitally collected. The models generated are the basis, for example, for digital twins of a real store, a web store (see the link to the demo website of a virtual store in the info box below) or automated product analyses, such as those carried out in laboratories.

    In addition, a scalable object recognition solution optimized for retail has been created, enabling automation solutions for production and logistics through precise optical product identification. Application examples include robot-based bin picking or bin packing, i.e. the packing of goods of all types and shapes. The latter task, also known as “order picking” or “pick & pack”, in which goods are removed from crates or shelves and neatly packed into a target carton, is becoming increasingly important, especially in these times of booming online retail. The object recognition solution developed to support these processes can be taught fully automatically on digital 3D product models in less than an hour for each new product.

    Smart shelving and kiosk systems

    Another result from Fraunhofer IPA is a mobile pointer unit that “helps” to move goods. The solution is aimed at skilled workers in logistics and retail and makes it easier for them to stock shelves. Employees wear a small barcode ring scanner on their finger which registers a product or its box to be placed on the shelf. A moving spotlight then illuminates the location where the product needs to be placed based on a digital store twin. The application is particularly useful for small products as well as those that are very similar to one another and saves time (see video link below).

    The Smart Kiosk System is a sales solution that was developed and tested. It uses the micromarket concept in order to branch out from supermarkets to include additional sales channels in the form of canteens and break rooms. For this purpose, weight sensors and cameras are installed in a refrigerator. With the help of object recognition, they make it possible to automatically recognize hands and products as well as remove or deposit products. This is combined with personal identification via a user interface and connection to a billing system (e.g., in this project, to SAP through partner Nagarro). Consequently, product withdrawals can be automatically booked in the virtual shopping cart and later billed digitally. In addition, service personnel receive a message when products need to be replenished, as the kiosk system constantly updates the digital twin on the connected K4R cloud platform. The project work took place in consultation with practice partner Livello.

    Integration in the open-source platform

    Last but not least, the IPA experts also advised on how the solutions developed as part of the project could best be specified and integrated in the K4R open-source platform. The openly expandable platform provides functions and interfaces for connecting to various logistics systems, sensors, robots or the extensive software modules from the Robot Operating System (ROS) environment, as well as providing the necessary infrastructure for store and product networked digital twins.

    Fraunhofer IPA has been leading the European part of the ROS-Industrial consortium for years and has consequently acquired extensive experience in the development of powerful, modular open-source systems. In fact, in predecessor projects such as Service Robot Network (SeRoNet), one of the project goals was to make the results sustainably accessible beyond the end of the project. For SeRoNet, this was achieved through the Xito.one virtual marketplace, a start-up development, and the ROStooling development tools, which were also posted on GitHub and used as a knowledge base for the K4R platform.

    Project profile
    Sponsor: German Federal Ministry for Economic Affairs and Climate Action BMWK: funding code 01MK20001G
    Duration: 01 January 2020 to 31 December 2022
    Project partners: Nagarro Allgeier ES GmbH, Deutsches Forschungszentrum für Künstliche Intelligenz [German Research Center for Artificial Intelligence] GmbH, dm-drogerie markt GmbH + Co. KG, dmTECH GmbH, EHI Retail Institute GmbH, fortiss GmbH, Fraunhofer-Gesellschaft, neusta GmbH, neusta software development West GmbH, team neusta GmbH, Technical University of Munich, Ubimax GmbH, University of Bremen


    Wissenschaftliche Ansprechpartner:

    Jochen Lindermayr | Phone +49 711 970-1565 | jochen.lindermayr@ipa.fraunhofer.de
    Richard Bormann | Phone +49 711 970-1062 | richard.bormann@ipa.fraunhofer.de


    Weitere Informationen:

    http://github.com/knowledge4retail -> GitHub-Link
    http://www.object-twin.ai/ -> Online shop website
    http://knowledge4retail.org/ -> project website


    Bilder

    Object recognition enables a robot to recognize products on shelves and tidily pack them into boxes with the help of "bin packing".
    Object recognition enables a robot to recognize products on shelves and tidily pack them into boxes ...
    Rainer Bez
    Fraunhofer IPA


    Anhang
    attachment icon Press release as PDF

    Merkmale dieser Pressemitteilung:
    Journalisten
    Maschinenbau
    überregional
    Forschungsprojekte
    Englisch


     

    Object recognition enables a robot to recognize products on shelves and tidily pack them into boxes with the help of "bin packing".


    Zum Download

    x

    Hilfe

    Die Suche / Erweiterte Suche im idw-Archiv
    Verknüpfungen

    Sie können Suchbegriffe mit und, oder und / oder nicht verknüpfen, z. B. Philo nicht logie.

    Klammern

    Verknüpfungen können Sie mit Klammern voneinander trennen, z. B. (Philo nicht logie) oder (Psycho und logie).

    Wortgruppen

    Zusammenhängende Worte werden als Wortgruppe gesucht, wenn Sie sie in Anführungsstriche setzen, z. B. „Bundesrepublik Deutschland“.

    Auswahlkriterien

    Die Erweiterte Suche können Sie auch nutzen, ohne Suchbegriffe einzugeben. Sie orientiert sich dann an den Kriterien, die Sie ausgewählt haben (z. B. nach dem Land oder dem Sachgebiet).

    Haben Sie in einer Kategorie kein Kriterium ausgewählt, wird die gesamte Kategorie durchsucht (z.B. alle Sachgebiete oder alle Länder).