UC-Net: Uncertainty Inspired RGB-D Saliency Detection via Conditional Variational Autoencoders

By Jing Zhang, Deng-Ping Fan, Yuchao Dai, Saeed Anwar, F. Saleh, Tong Zhang, Nick Barnes. Computer Vision and Pattern Recognition, 2020

377 citations recorded by the source corpus.

In this paper, we propose the first framework (UCNet) to employ uncertainty for RGB-D saliency detection by learning from the data labeling process. Existing RGB-D saliency detection methods treat the saliency detection task as a point estimation problem, and produce a single saliency map following a deterministic learning pipeline. Inspired by the saliency data labeling process, we propose probabilistic RGB-D saliency detection network via conditional variational autoencoders to model human annotation uncertainty and generate multiple saliency maps for each input image by sampling in the latent space. With the proposed saliency consensus process, we are able to generate an accurate saliency map based on these multiple predictions. Quantitative and qualitative evaluations on six challenging benchmark datasets against 18 competing algorithms demonstrate the effectiveness of our approach in learning the distribution of saliency maps, leading to a new state-of-the-art in RGB-D saliency detection.

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/2b26f42248a9d5df233110a3780a81e9ccbdc150

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

1. Deng-Ping Fan
2. Ling Shao
3. Ming-Ming Cheng
4. Ge-Peng Ji
5. Jianbing Shen
6. Ali Borji
7. Keren Fu
8. Qijun Zhao

In Computer Vision and Pattern Recognition

1. Deng-Ping Fan
2. Ling Shao
3. Ming-Ming Cheng
4. Jianbing Shen
5. Wenguan Wang
6. Ge-Peng Ji
7. Ali Borji
8. Huchuan Lu

Citations in the corpus

• Inf-Net: Automatic COVID-19 Lung Infection Segmentation From CT Images (2020, methodology): “Besides, due to the limited size of dataset, we will use the Generative Adversarial Network (GAN) [82] or Conditional Variational Autoencoders (CVAE) [83] to synthesize more samples, which can be regarded as a form of data augmentation to enhance the segmentation performance.”
• SwinNet: Swin Transformer Drives Edge-Aware RGB-D and RGB-T Salient Object Detection (2022, neutral): “DRLF [90], SSF [43], SSMA [38], A2dele [46], UC-Net [91], JL-DCF [92], CoNet [44], DANet [81], EBFSP [93],CDNet”
• BBS-Net: RGB-D Salient Object Detection with a Bifurcated Backbone Strategy Network (2020, background): “Two recent famous works have also explored uncertainty [68], bilateral attention [73], graph neural network [48] and a joint learning strategy [25] and achieve good performance.”
• PraNet: Parallel Reverse Attention Network for Polyp Segmentation (2020, background): “further introduce four other metrics which are widely used in the field of object detection [7,31,8,11,37,38].”
• RGB-D salient object detection: A survey (2020, methodology): “A comprehensive evaluation for 24 representative RGB-D based SOD models, including LHM [51], ACSD [56], DESM [49], GP [50], LBE [57], DCMC [1], SE [2], CDCP [84], CDB [95], DF [52], PCF [92], CTMF [58], CPFP [53], TANet [99], AFNet [102], MMCI [55], DMRA [54], D3Net [3], SSF [4], A2dele [5], S2MA [6”
• Salient Object Detection via Integrity Learning (2021, background): “[60] proposed a probabilistic RGBD saliency detection network, which uses conditional variational autoencoders to model the prediction uncertainty and then fuses the saliency maps generated for each input image to produce the final prediction.”
• Camouflaged Object Detection (2020, methodology): “In future work, we plan to extend COD10K dataset to provide input of various forms, for example, RGB-D camouflage object detection (similar to RGB-D salient object detection [19,71,74]), among others.”
• JL-DCF: Joint Learning and Densely-Cooperative Fusion Framework for RGB-D Salient Object Detection (2020, background): “Since then, only a dozen deep models have been proposed, as summarized in [18, 73], leaving large room for further improvement in performance.”
• Concealed Object Detection (2021, methodology): “In our future work, we plan to extend our COD10K dataset to provide inputs of various forms, such as multi-view images (like RGB-D SOD [15], [104]), textual descriptions, video (similar to video salient object detection [99]), among others.”