Deeply Supervised Salient Object Detection with Short Connections

By Qibin Hou, Ming-Ming Cheng, Xiaowei Hu, Ali Borji, Zhuowen Tu, Philip H. S. Torr. 2016

1,446 citations recorded by the source corpus.

Recent progress on salient object detection is substantial, benefiting mostly from the explosive development of Convolutional Neural Networks (CNNs). Semantic segmentation and salient object detection algorithms developed lately have been mostly based on Fully Convolutional Neural Networks (FCNs). There is still a large room for improvement over the generic FCN models that do not explicitly deal with the scale-space problem. The Holistically-Nested Edge Detector (HED) provides a skip-layer structure with deep supervision for edge and boundary detection, but the performance gain of HED on saliency detection is not obvious. In this paper, we propose a new salient object detection method by introducing short connections to the skip-layer structures within the HED architecture. Our framework takes full advantage of multi-level and multi-scale features extracted from FCNs, providing more advanced representations at each layer, a property that is critically needed to perform segment detection. Our method produces state-of-the-art results on 5 widely tested salient object detection benchmarks, with advantages in terms of efficiency (0.08 seconds per image), effectiveness, and simplicity over the existing algorithms. Beyond that, we conduct an exhaustive analysis of the role of training data on performance. We provide a training set for future research and fair comparisons.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

SSUL: Semantic Segmentation with Unknown Label for Exemplar-based Class-Incremental Learning (2021, methodology): “For the saliency-map detector to generate the Unknown class label, we employed DSS [11] pretrained on MSRA-B dataset [22], which contains 5,000 labels for salient objects.”
PU-Net: Point Cloud Upsampling Network (2018, background): “Therefore, we propose to directly combine features from different levels and let the network learn the importance of each level [9, 32, 10].”
Visual Saliency Transformer (2021, methodology): “To this end, they introduced various attention models [37, 80, 7], multi-scale feature integration methods [24, 49, 16, 43], and multi-task learning frameworks [67, 77, 82, 69, 25].”
Stacked Cross Refinement Network for Edge-Aware Salient Object Detection (2019, methodology): “Then fully convolutional networks (FCNs) [23] based approaches [4, 15, 22, 24, 29, 30, 31, 35, 36, 37] further boost the development of salient object detection.”
Interactive Image Segmentation With First Click Attention (2020, background): “…deep learning techniques, the demand for mask-level annotations has increased dramatically in some ﬁelds, such as salient object detection [4, 9, 14, 23], semantic segmentation [34], instance segmentation [21, 35], camouﬂaged object detection [15], and image/video manipulation [17, 28, 48].”
Camouflaged Object Segmentation with Distraction Mining (2021, background): “Multi-level feature aggregation is explored for robust detection [27, 19, 63, 68].”
A Simple Pooling-Based Design for Real-Time Salient Object Detection (2019, methodology): “When the edge branch is not incorporated, training only takes less than 6 hours on a training set of 5,000 images, which is quite faster than most of the previous methods [24, 43, 28, 44, 45, 9].”
CNNs-Based RGB-D Saliency Detection via Cross-View Transfer and Multiview Fusion (2018, neutral): “Work in [31] and [35]”
EGNet: Edge Guidance Network for Salient Object Detection (2019, methodology): “Image GT Ours PiCANet [33] PAGR [63] DGRL [52] SRM [49] UCF [62] Amulet [61] DSS [17, 18] DHS [32]”
BASNet: Boundary-Aware Salient Object Detection (2019, methodology): “We compare our method with 15 state-of-the-art models, PiCANetR [39], BMPM [72], R3Net [6], PAGRN [76], RADF [19], DGRL [65], RAS [4], C2S [36], LFR [73], DSS [17], NLDF [41], SRM [64], Amulet [74], UCF [75], MDF [35].”
PiCANet: Learning Pixel-Wise Contextual Attention for Saliency Detection (2017, methodology): “We compare our saliency model against other 9 state-of-the-art models, namely, SRM [38], DSS [9], NLDF [26], Amulet [45], UCF [46], DHS [23], RFCN [37], DCL [19], and MDF [18].”
Self-Erasing Network for Integral Object Attention (2018, methodology): “Saliency cues [5, 2, 33, 10, 13] are also adopted by some methods as the initial heuristic cues.”
Progressive Attention Guided Recurrent Network for Salient Object Detection (2018, methodology): “Salient Object Detection methods can be categorized as conventional low-level hand-crafted features based [20, 4, 33, 15, 9, 38, 23, 21] and Convolutional Neural Networks driven [25, 13, 37, 14, 12, 16, 27, 18, 7, 35, 36, 28] approaches.”
Self-produced Guidance for Weakly-supervised Object Localization (2018, background): “Indeed, some previous works use low-level feature maps to learn object regions [36,37].”
Reverse Attention for Salient Object Detection (2018, methodology): “In addition, some time consuming post-processing skills are also applied for refinement, e.g., superpixel-based filter [10], fully connected conditional random field (CRF) [22,8,17].”
Salient Objects in Clutter: Bringing Salient Object Detection to the Foreground (2018, result): “As also noticed in [40], training on the ECSSD dataset (1,000) allows to achieve better results than other datasets (e.”