GMNet: Graded-Feature Multilabel-Learning Network for RGB-Thermal Urban Scene Semantic Segmentation

By Wujie Zhou, Jinfu Liu, Jingsheng Lei, Lu Yu, Jenq-Neng Hwang. IEEE Transactions on Image Processing, 2021

317 citations recorded by the source corpus.

Semantic segmentation is a fundamental task in computer vision, and it has various applications in fields such as robotic sensing, video surveillance, and autonomous driving. A major research topic in urban road semantic segmentation is the proper integration and use of cross-modal information for fusion. Here, we attempt to leverage inherent multimodal information and acquire graded features to develop a novel multilabel-learning network for RGB-thermal urban scene semantic segmentation. Specifically, we propose a strategy for graded-feature extraction to split multilevel features into junior, intermediate, and senior levels. Then, we integrate RGB and thermal modalities with two distinct fusion modules, namely a shallow feature fusion module and deep feature fusion module for junior and senior features. Finally, we use multilabel supervision to optimize the network in terms of semantic, binary, and boundary characteristics. Experimental results confirm that the proposed architecture, the graded-feature multilabel-learning network, outperforms state-of-the-art methods for urban scene semantic segmentation, and it can be generalized to depth data.

Subfields: Civil and Structural Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/b64a91e5903be4e4ff4268c8deedc3828e22d3f2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Wei Zhong
2. Zhongxuan Luo
3. Xinxin Hu
4. Long Ma
5. Rainer Stiefelhagen
6. Jinyuan Liu
7. Jiaming Zhang
8. Huayao Liu

Citations in the corpus

• CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation With Transformers (2022, methodology): “2% , clearly outperforming the previous best RGB-T methods ABMDRNet [11], FEANet [17], and GMNet [42].”
• Multi-interactive Feature Learning and a Full-time Multi-modality Benchmark for Image Fusion and Segmentation (2023, methodology): “Besides comparing with the newest fusion-based methods, we also conduct the evaluations with competitive dual-stream methods: GMNet [52], FEANet [53], EGFNet [33], ABMDRNet [32] and LASNet [54].”