Disparity Map Walther, Schmidt, Schricker, Junger, Bergmann, Notni, Mäder: Automatic Detection and Prediction of Discontinuities in Laser Beam Butt Welding Utilizing Deep Learning, Journal of Advanced Joining Processes, 2022
Disparity Map Junger, Notni: Optimisation of a stereo image analysis by densify the disparity map based on a deep learning stereo matching framework, SPIE-Conference „Dimensional Optical Metrology and Inspection for Practical Applications XI“, 2022 Orbbec Best Student Paper Award
SteamVizzard Räth, Sattler: StreamVizzard – An Interactive and Explorative Stream Processing Editor, ACM International Conference on Distributed and Event‐Based Systems (DEBS), 2022
Interactive Stream Processing Räth, Sattler: Interactive and Explorative Stream Processing, ACM International Conference on Distributed and Event‐Based Systems (DEBS), 2022


Loss Functions Aganian, Eisenbach, Wagner, Seichter, Gross: Revisiting Loss Functions for Person Re-Identification, ENNS International Conference on Artificial Neural Networks (ICANN), 2021
Example Domain Description Model Bedini, Maschotta, Zimmermann: A generative Approach for creating Eclipse Sirius Editors for generic Systems, IEEE Syscon, 2021
Visual Scene Analysis Eisenbach, Aganian, Köhler, Stephan, Schröter, Gross: Visual Scene Understanding for Enabling Situation-Aware Cobots, IEEE International Conference on Automation Science and Engineering (CASE), 2021
Motion Planning Müller, Stephan, Gross: MDP-based Motion Planning for Grasping in Dynamic Scenarios, European Conference on Mobile Robots (ECMR), 2021
Laserstrahlschweißen Schmidt, Junger, Schricker, Bergmann, Notni: Echtzeitfähige Ansätze zum Monitoring der dehnungsfeldbasierten Spaltentstehung und resultierender Nahtqualität beim Laserstrahlschweißen, Jenaer Lasertagung, 2021
ESANet Seichter, Köhler, Lewandowski, Wengefeld, Gross: Efficient RGB-D Semantic Segmentation for Indoor Scene Analysis, IEEE International Conference on Robotics and Automation (ICRA), 2021

Dominik Walther, Leander Schmidt, Klaus Schricker, Christina Junger, Jean Pierre Bergmann, Gunther Notni, Patrick Mäder
Journal of Advanced Joining Processes, 2022

Abstract: Laser beam butt welding of thin sheets of high-alloy steel can be really challenging due to the formation of joint gaps, affecting weld seam quality. Industrial approaches rely on massive clamping systems to limit joint gap formation. However, those systems have to be adapted for each individually component geometry, making them very cost-intensive and leading to a limited flexibility. In contrast, jigless welding can be a high flexible alternative to substitute conventionally used clamping systems. Based on the collaboration of different actuators, motions systems or robots, the approach allows an almost free workpiece positioning. As a result, jigless welding gives the possibility for influencing the formation of the joint gap by realizing an active position control. However, the realization of an active position control requires an early and reliable error prediction to counteract the formation of joint gaps during laser beam welding. This paper proposes different approaches to predict the formation of joint gaps and gap induced weld discontinuities in terms of lack of fusion based on optical and tactile sensor data. Our approach achieves 97.4 % accuracy for video-based weld discontinuity detection and a mean absolute error of 0.02 mm to predict the formation of joint gaps based on tactile length measurements by using inductive probes.

Christina Junger, Gunther Notni
SPIE-Conference „Dimensional Optical Metrology and Inspection for Practical Applications XI“ in Orlando, USA, 2022

Orbbec Best Student Paper Award

Abstract: Stereo vision is used in many application areas, such as robot-assisted manufacturing processes. Recently, many different efficient stereo matching algorithms based on deep learning have been developed to solve the limitations of traditional correspondence point analysis, among others. The challenges include texture-poor objects or non-cooperative objects. One of these end-to-end learning algorithms is the Adaptive Aggregation Network (AANet/AANet+), which is divided into five steps: feature extraction, cost volume construction, cost aggregation, disparity computation and disparity refinement. By combining different components, it is easy to create an individual stereo matching model. Our goal is to develop efficient learning methods for robot-assisted manufacturing processes for cross-domain data streams. The aim is to improve recognition tasks and process optimisation. To achieve this, we have investigated the AANet+ in terms of usability and efficiency on our own test-dataset with different measurement setups (passive stereo system). Input of the AANet+ are rectified stereo pairs of the test-dataset and a pre-trained model. Instead of generating our own training dataset, we used two pre-trained models based on the KITTI-2015 and SceneFlow datasets. Our research has shown that the pretrained model based on the Scene Flow dataset predicts disparities with better object delimination. Due to the Out-of-Distribution inputs, only reliable disparity predictions of the AANet are possible for test data sets with parallel measurement setup. We compared the results with two traditional stereo matching algorithms (SemiGlobal block matching and DAISY). Compared to the traditionally computed disparity maps, the AANet+ is able to robustly detect texture-poor objects and optically non-cooperative objects.

Timo Räth, Kai-Uwe Sattler
ACM International Conference on Distributed and Event‐Based Systems (DEBS), Kopenhagen 2022


Abstract: Processing continuous data streams is one of the hot topics of our time. A major challenge is the formulation of a suitable and efficient stream processing pipeline. This process is complicated by long restart times after pipeline modifications and tight dependencies on the actual data to process. To approach these issues, we have developed StreamVizzard – an interactive and explorative stream processing editor to simplify the pipeline engineering process. Our system allows to visually configure, execute, and completely modify a pipeline during runtime without any delay. Furthermore, an adaptive visualizer automatically displays the operator’s processed data and statistics in a comprehensible way and allows the user to explore the data and support his design decisions. After the pipeline has been finalized our system automatically optimizes the pipeline based on collected statistics and generates standalone runtime code for productive use at a targeted stream processing engine.

Timo Räth, Kai-Uwe Sattler
ACM International Conference on Distributed and Event‐Based Systems (DEBS), Kopenhagen 2022

Interactive Stream Processing

Abstract: Formulating a suitable stream processing pipeline for a particular use case is a complicated process that highly depends on the processed data and usually requires many cycles of refinement. By combining the advantages of visual data exploration with the concept of real-time modifiability of a stream processing pipeline we want to contribute an interactive approach that simplifies and enhances the process of pipeline engineering. As a proof of concept, a prototype has been developed that delivers promising results and allows to modify the parameters and structure of stream processing pipelines at a development stage in a matter of milliseconds. By utilizing collected data and statistics from this explorative intermediate stage we will automatically generate optimized runtime code for a standalone execution of the constructed pipeline.

Dustin Aganian, Markus Eisenbach, Joachim Wagner, Daniel Seichter, Horst-Michael Gross
ENNS International Conference on Artificial Neural Networks (ICANN), 2021

Loss Functions

Abstract: Appearance-based person re-identification is very challenging, i.a. due to changing illumination, image distortion, and differences in viewpoint. Therefore, it is crucial to learn an expressive feature embedding that compensates for changing environmental conditions. There are many loss functions available to achieve this goal. However, it is hard to judge which one is the best. In related work, the experiments are only performed on the same datasets, but the use of different setups and different training techniques compromises the comparability. Therefore, we compare the most widely used and most promising loss functions under identical conditions on three different setups. We provide insights into why some of the loss functions work better than others and what additional benefits they provide. We further propose sequential training as an additional training trick that improves the performance of most loss functions. In our conclusion, we provide guidance for future usage and research regarding loss functions for appearance-based person re-identification. Source code is available.

Francesco Bedini, Ralph Maschotta, Armin Zimmermann
IEEE Syscon, 2021

Example DomainDescription Model


Abstract: Model-Driven Engineering (MDE) is getting more and more important for modeling, analyzing, and simulating complicated systems. It can also be used for both documenting and generating source code, which is less error-prone than a manually written one. For defining a model, it is common to have a graphical representation that can be edited through an editor. Creating such an editor for a given domain may be a difficult task for first-time users and a tedious, repetitive, and error-prone task for experienced ones. This paper introduces a new automated flow to ease the creation of ready-to-use Sirius editors based on a model, graphically defined by the domain experts, which describe their domains‘ structure. We provide different model transformations to generate the required artifacts to obtain a fully-fledged Sirius editor based on a generated domain metamodel. The generated editor can then be distributed as an Eclipse application or as a collaborative web application. Thanks to this generative approach, it is possible to reduce the cost of refactoring the domain’s model in successive iterations, as only the final models need to be updated to conform to the latest format. At the same time, the editor gets generated and hence updated automatically at practically no cost.

Markus Eisenbach, Dustin Aganian, Mona Köhler, Benedict Stephan, Christof Schröter, Horst-Michael Gross
IEEE International Conference on Automation Science and Engineering (CASE), 2021

Visual Scene Analysis

Abstract: Although in the course of Industry 4.0, a high degree of automation is the objective, not every process can be fully automated – especially in versatile manufacturing. In these applications, collaborative robots (cobots) as helpers are a promising direction. We analyze the collaborative assembly scenario and conclude that visual scene understanding is a prerequisite to enable autonomous decisions by cobots. We identify the open challenges in these visual recognition tasks and propose promising new ideas on how to overcome them.

Steffen Müller, Benedict Stephan, Horst-Michael Gross
European Conference on Mobile Robots (ECMR), 2021

Motion Planning

Abstract: Path planning for robotic manipulation is a well understood topic as long as the execution of the plan takes place in a static scene. Unfortunately, for applications involving human interaction partners a dynamic obstacle configuration has to be considered. Furthermore, if it comes to grasping objects from a human hand, there is not a single goal position and the optimal grasping configuration may change during the execution of the grasp movement. This makes a continuous replanning in a loop necessary. Besides efficiency and security concerns, such periodic planning raises the additional requirement of consistency, which is hard to achieve with traditional sampling based planners. We present an online capable planner for continuous control of a robotic grasp task. The planner additionally is able to resolve multiple possible grasp poses and additional goal functions by applying an MDP-like optimization of future rewards. Furthermore, we present a heuristic for setting edges in a probabilistic roadmap graph that improves the connectivity and keeps edge count low.

Leander Schmidt, Christina Junger, Klaus Schricker, Jean Pierre Bergmann, Gunther Notni
DVS Berichte 367 zur Jenaer Lasertagung, S. 43-54, 2021

Abstract: Das Laserstrahlschweißen bedingt aufgrund des lokalen Aufschmelz- und Erstarrungsprozesses die Entstehung eines dehnungsfeldbasierten Fügespaltes. Industrielle Lösungsansätze setzen zur Begrenzung dieses Phänomens massive Spannsysteme ein, welche sehr kostenintensiv sind und zugleich bei Änderungen der Bauteilgeometrie individuell angepasst werden müssen. Demgegenüber bietet sich zur Flexibilisierung der Probeneinspannung der Einsatz des vorrichtungsfreien Schweißens (engl. jigless welding) an. Erste Ansätze für das lichtbogenbasierte Schweißen demonstrieren hierbei das hohe Potential zur in-Prozess-Anpassung des Fügespaltes unter Einsatz kollaborativer Robotik. Auf Grundlage abweichender Verfahrensanforderungen (insbesondere Positionstoleranz) sind diese Ansätze bislang nicht für das Laserstrahlschweißen umgesetzt. Diesbezüglich fehlen insbesondere Lösungsansätze, welche ein echtzeitfähiges Monitoring der Nahtqualität in Abhängigkeit des Fügespalts sowie des Winkelverzugs der Bleche erlauben. Diese Veröffentlichung zeigt daher erste Ansätze zur Bewertung der Nahtqualität von laserstrahlgeschweißten Feinblechen des Werkstoffs X5CrNi18–10/1.4301 in Abhängigkeit des Fügespalts sowie des Winkelverzugs einer I-Naht am Stumpfstoß. Durch Variation der Schweißgeschwindigkeit (1/5/10 m/min) sowie der Blechdicke (0,5/1/2 mm) wurden verschiedene Charakteristika erfasst und im Verhältnis zur resultierenden Nahtqualität bewertet. Auf Basis einer multimodalen Datenanalyse wurden mögliche Regelungsgrößen evaluiert, welche eine vielversprechende Ausgangsbasis zur Umsetzung einer echtzeitfähigen Prozessregelung bieten.

Daniel Seichter, Mona Köhler, Benjamin Lewandowski, Tim Wengefeld, Horst-Michael Gross
IEEE International Conference on Robotics and Automation (ICRA), 2021


Abstract: Analyzing scenes thoroughly is crucial for mobile robots acting in different environments. Semantic segmentation can enhance various subsequent tasks, such as (semantically assisted) person perception, (semantic) free space detection, (semantic) mapping, and (semantic) navigation. In this paper, we propose an efficient and robust RGB-D segmentation approach that can be optimized to a high degree using NVIDIA TensorRT and, thus, is well suited as a common initial processing step in a complex system for scene analysis on mobile robots. We show that RGB-D segmentation is superior to processing RGB images solely and that it can still be performed in real time if the network architecture is carefully designed. We evaluate our proposed Efficient Scene Analysis Network (ESANet) on the common indoor datasets NYUv2 and SUNRGB-D and show that we reach state-of-the-art performance while enabling faster inference. Furthermore, our evaluation on the outdoor dataset Cityscapes shows that our approach is suitable for other areas of application as well. Finally, instead of presenting benchmark results only, we also show qualitative results in one of our indoor application scenarios.