Progressively Complementarity-Aware Fusion Network for RGB-D Salient Object Detection

By Hao Chen, Youfu Li. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018

303 citations recorded by the source corpus.

How to incorporate cross-modal complementarity sufficiently is the cornerstone question for RGB-D salient object detection. Previous works mainly address this issue by simply concatenating multi-modal features or combining unimodal predictions. In this paper, we answer this question from two perspectives: (1) We argue that if the complementary part can be modelled more explicitly, the cross-modal complement is likely to be better captured. To this end, we design a novel complementarity-aware fusion (CA-Fuse) module when adopting the Convolutional Neural Network (CNN). By introducing cross-modal residual functions and complementarity-aware supervisions in each CA-Fuse module, the problem of learning complementary information from the paired modality is explicitly posed as asymptotically approximating the residual function. (2) Exploring the complement across all the levels. By cascading the CA-Fuse module and adding level-wise supervision from deep to shallow densely, the cross-level complement can be selected and combined progressively. The proposed RGB-D fusion network disambiguates both cross-modal and cross-level fusion processes and enables more sufficient fusion results. The experiments on public datasets show the effectiveness of the proposed CA-Fuse module and the RGB-D salient object detection network.

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/a56ff3c9a84286a51a56023300e9d95e1dbebdd0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

Citations in the corpus

JL-DCF: Joint Learning and Densely-Cooperative Fusion Framework for RGB-D Salient Object Detection (2020, methodology): “Generally, as summarized by previous literature [4, 77], the above approaches can be divided into three categories: (a) Early-fusion [18,31,40,55], (b) late-fusion [26,60] and (c) middle-fusion [4–6 , 80 ].”
Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks (2019, methodology): “Some works [49], [51], [61] use the same F-”
BBS-Net: RGB-D Salient Object Detection with a Bifurcated Backbone Strategy Network (2020, methodology): “d various network architectures. For example, [46] fed a four-channel RGB-D image into a single backbone and then obtained saliency map outputs from each sideout features (single stream). Chen et al. [3] leveraged two networks to extract RGB features and depth features respectively, and then fused t”
Contrast Prior and Fluid Pyramid Integration for RGBD Salient Object Detection (2019, methodology): “Here we use the VGG-16 suggested in [4] for fair comparison and the last three layers are truncated.”
An Underwater Image Enhancement Benchmark Dataset and Beyond (2019, background): “Resizing the training data to a fixed size is widely used in deep learning, such as image dehazing [66], salient object detection [70], etc.”
Suppress and Balance: A Simple Gated Network for Salient Object Detection (2020, methodology): “etwork pipeline. A.2 RGB-D Salient object detection Dataset. There are ve main RGB-D SOD datasets which are NJUD [23], RGBD135 [9] NLPR [34], SSD [74] and SIP [16]. We adopt the same splitting way as [3,5,18,71,36] to guarantee a fair comparison. We split 1,485 samples from NJUD and 700 samples from”
Cross-level Feature Aggregation Network for Polyp Segmentation (2023, background): “Several works have also been developed to study the integration of multi-level features in the ﬁeld of saliency detection and camouﬂaged object detection [31–34] .”
RGB-D salient object detection: A survey (2020, methodology): “A comprehensive evaluation for 24 representative RGB-D based SOD models, including LHM [51], ACSD [56], DESM [49], GP [50], LBE [57], DCMC [1], SE [2], CDCP [84], CDB [95], DF [52], PCF [92], CTMF [58], CPFP [53], TANet [99], AFNet [102], MMCI [55], DMRA [54], D3Net [3], SSF [4], A2dele [5], S2MA [6”
Review of Visual Saliency Detection With Comprehensive Information (2018, methodology): “Considering the ways of using depth information, we divide the RGBD saliency detection model into depth feature based method [55]–[62] and depth measure based method [63]–[70].”
ACNET: Attention Based Network to Exploit Complementary Features for RGBD Semantic Segmentation (2019, background): “The other like [5][9] just fuses the RGBD features at the downsampling stage.”
Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a review (2020, background): “MaskLab Chen et al.”