FuseNet: Incorporating Depth into Semantic Segmentation via Fusion-Based CNN Architecture

By Caner Hazirbas, Lingni Ma, Csaba Domokos, Daniel Cremers. 2016

865 citations recorded by the source corpus.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

AdaBins: Depth Estimation Using Adaptive Bins (2020, background): “This is a classical problem in computer vision that is essential for many applications [26, 30, 16, 6].”
High Quality Monocular Depth Estimation via Transfer Learning (2018, background): “Depth estimation from 2D images is a fundamental task in many applications including scene understanding and reconstruction [24, 29, 15].”
Depth-aware CNN for RGB-D Segmentation (2018, background): “In [14,15,16,17], depth is simply treated as additional channels and directly fed into CNN.”
3D Shape Segmentation with Projective Convolutional Networks (2016, background): “Recent works [14, 45, 18] have applied a similar strategy for indoor scene recognition tasks.”
Unsupervised Reverse Domain Adaptation for Synthetic Medical Images via Adversarial Training (2017, background): “Depth information has been shown to improve performance of these tasks [41], but depth measurement remains challenging through an endoscope because of unpredictable movement, small working distances, limited endoscope size, non-uniform tissue texture, and the”
A Fully Convolutional Two-Stream Fusion Network for Interactive Image Segmentation (2018, methodology): “Also, we note that another possible way to fuse the image and user interaction features is the layer-by-layer fusion proposed by Hazirbas et al. [29].”
What Makes Multimodal Learning Better than Single (Provably) (2021, background): “Inspired from the success of deep learning [26, 21], deep multi-modal research is also activated, which covers fields like audio-visual learning [8, 43], RGB-D semantic segmentation [35, 20] and Visual Question Answering [17, 2].”
MMTM: Multimodal Transfer Module for CNN Fusion (2019, background): “This approach is closely related to residual learning [51] and has been proposed for multimodal fusion with aligned spatial dimensions [77].”
SnapNet: 3D point cloud semantic labeling with 2D deep segmentation networks (2017, methodology): “In order to face the class imbalance problem of SUN RGBD dataset we had to weight each class participation over the loss function as in [48].”
Dynamic Fusion Module Evolves Drivable Area and Road Anomaly Detection: A Benchmark and Algorithms (2021, methodology): “Different from FuseNet, the depth-aware CNN [11] introduces two novel operations: 1) depth-aware convolution and 2) depth-aware average pooling.”
RDFNet: RGB-D Multi-level Residual Feature Fusion for Indoor Semantic Segmentation (2017, methodology): “(a) early fusion, (b) late fusion, (c) the architecture proposed by [39], (d) the architecture proposed by [13], where ‘C’, ‘T’, and ‘+’ represent the concatenation, transformation, and element-wise summation, respectively.”
RTFNet: RGB-Thermal Fusion Network for Semantic Segmentation of Urban Scenes (2019, methodology): “[23] proposed FuseNet using both the RGB and depth images provided by RGB-D cameras [24]–[26].”
A survey on deep learning techniques for image and video semantic segmentation (2018, methodology): “The proposed approach is based on FuseNet[114], which combines RGB and depth images for semantic segmentation, and improves the original work by adding multi-scale loss minimization.”
Self-Supervised Drivable Area and Road Anomaly Segmentation Using RGB-D Data For Robotic Wheelchairs (2019, methodology): “Recent development of deep learning techniques has brought significant improvements on the semantic segmentation using RGB-D cameras [5]–[7].”
Beyond RGB: Very High Resolution Urban Remote Sensing With Multimodal Deep Networks (2017, neutral): “(a) FuseNet architecture [10] for early fusion of remote sensing data.”