MFNet: Towards real-time semantic segmentation for autonomous vehicles with multi-spectral scenes

By Qishen Ha, Kohei Watanabe, Takumi Karasawa, Yoshitaka Ushiku, Tatsuya Harada. 2017

668 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Industrial and Manufacturing Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Yiannis Demiris
2. Xingchen Zhang
3. Jiayi Ma
4. Linfeng Tang
5. Yong Ma
6. Jun Huang
7. Hao Zhang
8. Xiaoguang Mei

In Industrial and Manufacturing Engineering

1. Karthik Ramani
2. Ming Liu
3. Klaus C. J. Dietmayer
4. Rainer Stiefelhagen
5. Yuxiang Sun
6. Jiayi Ma

Citations in the corpus

• A Survey of Autonomous Driving: Common Practices and Emerging Technologies (2019, background): “Other camera types gaining interest for ADS include flash cameras [78], thermal cameras [80], [81], and”
• nuScenes: A Multimodal Dataset for Autonomous Driving (2019, neutral): “cities Vistas [26] 2017 - - 25k 0 0 25k 0 Yes Yes Yes 6 continents BDD100k [60] 2017 100k 1k 100M 0 0 100k 0 Yes Yes Yes NY, SF M. obj. det. [51] 2017 - - 7.5k 0 0 7.5k 0 Yes No No Tokyo M. sem. seg. [29] 2017 - - 1.6k 0 0 1.6k 0 Yes No No - ApolloScape [33] 2018 - 100 144k 0 0 144k 70k Yes No No 4x”
• SwinFusion: Cross-domain Long-range Learning for General Image Fusion via Swin Transformer (2022, methodology): “The MSRS dataset [76], [77]1, containing 1,083 training image pairs and 361 testing image pairs, is selected for training and evaluating the visible and infrared image fusion task.”
• SDNet: A Versatile Squeeze-and-Decomposition Network for Real-Time Image Fusion (2021, methodology): “24 The proposed method cannot handle non-registered data from Ha et al. (2017), and artifacts appear in the fused results not handle these real unregistered data, as shown in Fig.”
• RTFNet: RGB-Thermal Fusion Network for Semantic Segmentation of Urban Scenes (2019, methodology): “MFNet was proposed by Ha et al. [27] for semantic segmentation of urban scenes using both the visible and thermal cameras.”
• SNE-RoadSeg: Incorporating Surface Normal Information into Semantic Segmentation for Accurate Freespace Detection (2020, methodology): “4: Examples of the experimental results on (a) the SYNTHIA road dataset and (b) our R2D road dataset: (i) RGB, (ii) Depth, (iii) SNE-Depth (Ours), (iv) RGBD and (v) SNE-RGBD (Ours); (1) DeepLabv3+ [9], (2) U-Net [26], (3) SegNet [3], (4) GSCNN [30], (5) DUpsampling [32], (6) DenseASPP [37], (7) Fuse”
• Improving Multispectral Pedestrian Detection by Addressing Modality Imbalance Problems (2020, background): “Recent years have witnessed increasing researches towards object detection among vision community by taking the advantages of multi-modal inputs, such as RGB + thermal, RGB + depth, RGB + LiDAR and so on [19,17,15,1].”
• SuperFusion: A Versatile Image Registration and Fusion Network with Semantic Awareness (2022, background): “The complementary nature of visible and infrared images encourages researchers to integrate useful information from multi-modal images into a single fused image to facilitate human perception and subsequent visual applications, such as object detection, object tracking, semantic segmentation, securi”
• Dynamic Fusion Module Evolves Drivable Area and Road Anomaly Detection: A Benchmark and Algorithms (2021, methodology): “Performance comparison among four data-fusion networks (FuseNet [28], MFNet [30], depth-aware CNN [11], and RTFNet [29]) and our DFMRTFNet with five training data setups on our GMRP dataset, where D-A CNN is short for depth-aware CNN, and the best result is highlighted in each subfigure.”
• Visible and Infrared Image Fusion Using Deep Learning (2023, methodology): “Some VIF datasets, including TNO [176], INO [177], MFNet [178], RoadScene [61], VIFB [167], LLVIP [179] and M3FD [115], have been utilized in the VIF task as test sets to evaluate image fusion performance.”
• Multi-interactive Feature Learning and a Full-time Multi-modality Benchmark for Image Fusion and Segmentation (2023, methodology): “MFNet FMB Figure 6: Comparison of FMB with existing multi-modality segmentation datasets ( i.e., PST900 [31] and MFNet [30]).”
• CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation With Transformers (2022, methodology): “objects through their specific infrared imaging [10], [11].”
• Multimodal End-to-End Autonomous Driving (2019, methodology): “on multimodal data [39], [40]); monocular depth estimation [41], [42], [43], [44]; stixel-world representation (based on stereo images [45], [46], [47], [48], monocular [49], [50], and multimodal [51]); as well as SLAM and place recognition [52], [53], [54], [55], [56], [57], [58], [59], [60], [61].”
• Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges (2019, background): “IV-A, there are fewer works on multi-modal semantic segmentation: [91], [118], [123] employ RGB and thermal images, [60] fuses RGB images and depth images from a stereo camera, [124]–[126] combine RGB, thermal, and depth images for semantic segmentation in diverse environments such as forests, [122]”
• First-Person View Hand Segmentation of Multi-Modal Hand Activity Video Dataset (2020, methodology): “MFNet [23] showed that fusing LWIR and RGB frames sig-nificantly enhances the segmentation performance.”