Salient Object Detection in the Deep Learning Era: An In-Depth Survey

By Wenguan Wang, Qiuxia Lai, Huan Fu, Jianbing Shen, Haibin Ling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019

747 citations recorded by the source corpus.

As an essential problem in computer vision, salient object detection (SOD) has attracted an increasing amount of research attention over the years. Recent advances in SOD are predominantly led by deep learning-based solutions (named deep SOD). To enable in-depth understanding of deep SOD, in this paper, we provide a comprehensive survey covering various aspects, ranging from algorithm taxonomy to unsolved issues. In particular, we first review deep SOD algorithms from different perspectives, including network architecture, level of supervision, learning paradigm, and object-/instance-level detection. Following that, we summarize and analyze existing SOD datasets and evaluation metrics. Then, we benchmark a large group of representative SOD models, and provide detailed analyses of the comparison results. Moreover, we study the performance of SOD algorithms under different attribute settings, which has not been thoroughly explored previously, by constructing a novel SOD dataset with rich attribute annotations covering various salient object types, challenging factors, and scene categories. We further analyze, for the first time in the field, the robustness of SOD models to random input perturbations and adversarial attacks. We also look into the generalization and difficulty of existing SOD datasets. Finally, we discuss several open issues of SOD and outline future research directions. All the saliency prediction maps, our constructed dataset with annotations, and codes for evaluation are publicly available at https://github.com/wenguanwang/SODsurvey.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/0167e98f6d2e4c44b505c0f74f91425f62dfc62c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators (2023, methodology): “In more detail, given the generated sequence of our video generator, x 1: m 0 , we apply (an in-house solution for) salient object detection [40] to the decoded images to obtain a corresponding foreground mask M k for each frame k .”
Deep semantic segmentation of natural and medical images: a review (2019, methodology): “Weng et al. (2019a) applied a neural architecture search method to U-Net to obtain a smaller network with a better organ/tumor segmentation performance on CT, MR, and ultrasound images.”
JL-DCF: Joint Learning and Densely-Cooperative Fusion Framework for RGB-D Salient Object Detection (2020, methodology): “(i) Deep-based RGB-D SOD methods are still underexplored: Despite more than one hundred papers on RGB SOD models being published since 2015 [15,61,64,65,69], there are only a few deep learning-based works focusing on RGB-D SOD.”
Multi-Scale Interactive Network for Salient Object Detection (2020, background): “See [2, 35] for more details about traditional and early deep methods.”
Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a review (2020, background): “Readers could refer to some surveys [Borji et al., 2019; Wang et al., 2019e] for comprehensive understandings of 2D salient object detection.”
Spatiotemporal Knowledge Distillation for Efficient Estimation of Aerial Video Saliency (2019, background): “The recent advances in the field of saliency estimation from videos result in many visual saliency models [23].”
F3Net: Fusion, Feedback and Focus for Salient Object Detection (2019, background): “Salient object detection (SOD) aims to estimate the visual signiﬁcant regions of images or videos and often serves as the pre-processing step for many downstream vision tasks (Wang et al. 2019a).”
Camouflaged Object Detection (2020, methodology): “(a) Image (b) Generic object (c) Salient object (d) Camouflaged object Figure 2: Given an input image (a), we present the ground-truth for (b) panoptic segmentation [30] (which detects generic objects [39,44] including stuff and things), (c) salient instance/object detection [16, 33, 61, 76] (which ”
Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks (2019, methodology): “Salient object detection (SOD) methods [1]– [17] have been widely used in taking perfect portraits by automatically adding large aperture and other enhancement effects.”
Label Decoupling Framework for Salient Object Detection (2020, neutral): “procedure in downstream computer vision tasks [29, 35].”
Interactive Two-Stream Decoder for Accurate and Fast Saliency Detection (2020, background): “For other researches please refer the recent survey [2, 6, 34, 23, 33] for more details.”
Visual Saliency Transformer (2021, methodology): “Hence, CNNbased methods have achieved impressive results [66, 88].”
Concealed Object Detection (2021, methodology): “We refer readers to recent survey and benchmark papers [11], [34], [35], [36] for more details.”