idw – Informationsdienst Wissenschaft
Nachrichten, Termine, Experten
Nachrichten, Termine, Experten
Generative AI like ChatGPT can write text, generate images, and even write code. But when it comes to the physical world, these models reach their limits. They have only a limited understanding of spaces and changes in the environment. The team at the startup SE3 Labs has developed a spatial AI technology that uses images, videos, and map data to create realistic 3D models. The technology is already being used in numerous projects by the German Armed Forces.
It all began at a climbing gym. That’s where Daniel Cremers, professor for Computer Vision and Artificial Intelligence at the Technical University of Munich (TUM), met his future co-founder, Lukas Köstler. Shortly thereafter, Lukas Köstler began conducting research as a doctoral student in Professor Cremers’ department; Simon Klenk also joined the team there. As a result of this collaboration, the three decided to continue their research in their own company. With SE3 Labs, the team aims to give machines a spatial understanding of their environment so that they can act independently within it - similar to how generative AI handles language. Their startup receives support from UnternehmerTUM and TUM Venture Labs.
From visual data to spatial understanding
“Our technology is designed not only to recognize what is visible in an image or video, but also to truly understand the three-dimensional world - that is, how objects, buildings, streets, or terrain features relate to one another spatially,” says Daniel Cremers. Autonomous systems such as drones or robots can use this spatial understanding to orient themselves and act independently in their environment. In addition, the digital representation of the environment can be analyzed and queried by users.
The software processes sensor data in real time to create a continuously updated, three-dimensional model of the environment. Terrain profiles, elevation differences, and relevant objects are captured and localized with high accuracy. This combines methods from large language models with computer vision - that is, techniques that enable machines to recognize and understand visual information in a manner similar to humans.
Users can ask questions about this spatial data, just as they would with a chatbot. In urban planning, for example, the tool can help determine how many solar panels could fit on existing rooftops or how many parking spaces are available in a specific neighborhood.
Focus on deployment in crisis and war zones
“Our technology is particularly relevant, however, in situations where up-to-date information about the surroundings is crucial, such as in crisis and war zones. Autonomous systems like drones often cannot be deployed in these cases because they rely on GPS signals, which are frequently unavailable in these areas,” says Lukas Köstler.
This is exactly where the SE3 Labs team comes in, as SE3’s navigation system enables precise autonomous navigation even in environments without a GPS signal. The system continuously creates its own spatial map of the surroundings, allowing it to account for changes in the terrain or operational area in real time.
Using the same 3D map, emergency responders can then ask specific questions: How many of certain objects are there in an area? Which roads are passable? How has the environment changed since the last survey? The technology is designed to help analyze large and complex areas more quickly and accurately. In the long term, SE3 Labs sees potential applications in many fields: from defense and security to smart infrastructure and construction, all the way to industrial applications.
Daniel Cremers
Technical University of Munich
Professor for Computer Vision and Artificial Intelligence
cremers@tum.de
https://www.tum.de/en/news-and-events/all-news/press-releases/details/how-autono...
Merkmale dieser Pressemitteilung:
Journalisten
Informationstechnik
überregional
Forschungs- / Wissenstransfer, Forschungsprojekte
Englisch
Sie können Suchbegriffe mit und, oder und / oder nicht verknüpfen, z. B. Philo nicht logie.
Verknüpfungen können Sie mit Klammern voneinander trennen, z. B. (Philo nicht logie) oder (Psycho und logie).
Zusammenhängende Worte werden als Wortgruppe gesucht, wenn Sie sie in Anführungsstriche setzen, z. B. „Bundesrepublik Deutschland“.
Die Erweiterte Suche können Sie auch nutzen, ohne Suchbegriffe einzugeben. Sie orientiert sich dann an den Kriterien, die Sie ausgewählt haben (z. B. nach dem Land oder dem Sachgebiet).
Haben Sie in einer Kategorie kein Kriterium ausgewählt, wird die gesamte Kategorie durchsucht (z.B. alle Sachgebiete oder alle Länder).
