Efficient RGB-D Semantic Segmentation for Indoor Scene Analysis

By Daniel Seichter, Mona Köhler, Benjamin Lewandowski, Tim Wengefeld, Horst-Michael Groß. IEEE International Conference on Robotics and Automation, 2020

316 citations recorded by the source corpus.

Analyzing scenes thoroughly is crucial for mobile robots acting in different environments. Semantic segmentation can enhance various subsequent tasks, such as (semantically assisted) person perception, (semantic) free space detection, (semantic) mapping, and (semantic) navigation. In this paper, we propose an efficient and robust RGB-D segmentation approach that can be optimized to a high degree using NVIDIA TensorRT and, thus, is well suited as a common initial processing step in a complex system for scene analysis on mobile robots. We show that RGB-D segmentation is superior to processing RGB images solely and that it can still be performed in real time if the network architecture is carefully designed. We evaluate our proposed Efficient Scene Analysis Network (ESANet) on the common indoor datasets NYUv2 and SUNRGB-D and show that we reach state-of-the-art performance while enabling faster inference. Furthermore, our evaluation on the outdoor dataset Cityscapes shows that our approach is suitable for other areas of application as well. Finally, instead of presenting benchmark results only, we also show qualitative results in one of our indoor application scenarios.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/5c784da61c7ef70c2f8f61533aaca391ab067e5e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation With Transformers (2022, background): “S EMANTIC segmentation is an essential task in computer vision, which aims to transform an image input into its underlying semantically meaningful regions and enables a pixelwise dense scene understanding for many real-world applications such as automated vehicles, robotics navigation, and augmented”
Image Segmentation Using Deep Learning: A Survey (2020, methodology): “Network (UperNet) [136], segmentation by re-training and self-training [137], densely connected neural architecture search [138], hierarchical multiscale attention [139], Efficient RGB-D Semantic Segmentation (ESA-Net) [140], Iterative Pyramid Contexts [141], and Learning Dynamic Routing for Semanti”
A Multilevel Multimodal Fusion Transformer for Remote Sensing Semantic Segmentation (2024, methodology): “…benchmarked the performance of the proposed FTransUNet against twelve representative state-of-the-art methods, namely ABCNet [61], PSPNet [62], MAResU-Net [63], vFuseNet [15], FuseNet [14], ESANet [33], Tran-sUNet [31], SA-GATE [64], UNetFormer [44], CMFNet [19], MFTransNet [59], and CMGFNet [47].”
What Makes Multimodal Learning Better than Single (Provably) (2021, background): “Inspired from the success of deep learning [26, 21], deep multi-modal research is also activated, which covers fields like audio-visual learning [8, 43], RGB-D semantic segmentation [35, 20] and Visual Question Answering [17, 2].”