DeepSaliency: Multi-Task Deep Neural Network Model for Salient Object Detection

By Xi Li, Liming Zhao, Lina Wei, Ming-Hsuan Yang, Fei Wu, Yueting Zhuang, Haibin Ling, Jingdong Wang. IEEE Transactions on Image Processing, 2015

511 citations recorded by the source corpus.

A key problem in salient object detection is how to effectively model the semantic properties of salient objects in a data-driven manner. In this paper, we propose a multi-task deep saliency model based on a fully convolutional neural network with global input (whole raw images) and global output (whole saliency maps). In principle, the proposed saliency model takes a data-driven strategy for encoding the underlying saliency prior information, and then sets up a multi-task learning scheme for exploring the intrinsic correlations between saliency detection and semantic image segmentation. Through collaborative feature learning from such two correlated tasks, the shared fully convolutional layers produce effective features for object perception. Moreover, it is capable of capturing the semantic information on salient objects across different levels using the fully convolutional layers, which investigate the feature-sharing properties of salient object detection with a great reduction of feature redundancy. Finally, we present a graph Laplacian regularized nonlinear regression model for saliency refinement. Experimental results demonstrate the effectiveness of our approach in comparison with the state-of-the-art approaches.

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/877628547f227c874d20d28a36d6de11d9dadde2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

Citations in the corpus

Deeply Supervised Salient Object Detection with Short Connections (2016, methodology): “We compare the proposed approach with 7 recent CNNbased methods, including MDF [47], DS [60], DCL [35], ELD [50], MC [49], RFCN [51], and DHS [36].”
Learning Uncertain Convolutional Features for Accurate Saliency Detection (2017, methodology): “Second, instead of simply using FCNs as predictors in [26, 45], our model can learn uncertain convolutional features by using multiple reformulated dropouts, which improve the robustness and accuracy of saliency detection.”
Object Detection With Deep Learning: A Review (2018, background): “[149], MTDNN [141], CRPSD [142], DCL [143], ELD [153], NLDF [154] and DSSC [155].”
Salient Object Detection: A Discriminative Regional Feature Integration Approach (2013, methodology): “2015), ii MDF (Li and Yu 2015), iii DS (Li et al. 2015), iv DCL (Li and Yizhou 2016)”
Learning to Detect Salient Objects with Image-Level Supervision (2017, methodology): “Recently, FCN based saliency methods [29, 49] have been proposed, with more competitive performance in terms of both accuracy and speed.”
R³Net: Recurrent Residual Refinement Network for Saliency Detection (2018, background): “To improve the detection accuracy, many deep saliency networks [Li and Yu, 2015; Zhao et al., 2015; Li et al., 2016] have been proposed by leveraging the deep semantic features of fully convolutional neural networks (FCNs) [Long et al., 2015].”
Salient Object Detection in the Deep Learning Era: An In-Depth Survey (2019, methodology): “• DS [72] consists of an SOD branch and a semantic segmentation branch sharing the bottom layers for extracting the semantically rich features.”
Progressive Attention Guided Recurrent Network for Salient Object Detection (2018, methodology): “BL [23], KSR [29], LEGS [25], ELD [12], MDF [13], DS [16], MCDL [37], DCL [14], RFCN [27], DHS [18], UCF [36] and Amulet [35].”
RGBD Salient Object Detection via Deep Fusion (2016, methodology): “To better capture the object boundaries, various postprocessing approaches are used to refine the outputs of the FCNs, such as CRF [48], [49], regularized nonlinear regression [46], and edge-aware erosion [45].”
From Discriminant to Complete: Reinforcement Searching-Agent Learning for Weakly Supervised Object Detection (2020, methodology): “In future works, we will explore the potential of using the saliency detection technique [1], [58], [59] to obtain the better local discriminative regions and apply the more advanced reinforcement learning algorithms [60], [61] to further improve the learning performance of the proposed WSOD…”
Video Salient Object Detection via Fully Convolutional Networks (2017, unknown): “and more methods [34], [36]–[38] tend to learn in an end-toend manner and directly generate pixel-wise saliency maps via fully convolutional networks (FCNs) [5].”
Salient Objects in Clutter: Bringing Salient Object Detection to the Foreground (2018, methodology): “The DS [32] model set up a multi-task learning scheme for exploring the intrinsic correlations between saliency detection and semantic image segmentation, which shared the information in FCN layers to generate effective features for object perception.”
Deep Level Sets for Salient Object Detection (2017, methodology): “MTDS solves an optimization problem for every input based on superpixels and coarse saliency map output by a fully convolutional neural network, and takes about 5.6 seconds per image (implemented in python/caffe).”
Salient Object Detection With Pyramid Attention and Salient Edges (2019, methodology): “We compare the proposed PAGE-Net against 19 recent deep learning based alternatives: MDF [21], LEGS [34], DS [24], DCL [22], ELD [20], MC [57], RFCN [36], DHS [26], HEDS [14], KSR [38], NLDF [29], DLS [15], AMU [54], UCF [55], SRM [37], FSN [8], PAGR [56], RAS [7] and C2S [23].”
A Stagewise Refinement Model for Detecting Salient Objects in Images (2017, methodology): “We compare our method with eleven state-of-the-art deep learning-based and classic saliency detection methods, including DRFI [24], BL [43], LEGS [46], MDF [28], MCDL [56], DS [31], DHS [33], ELD [26], DCL [29], KSR [49] and RFCN [48].”
Attentive Feedback Network for Boundary-Aware Salient Object Detection (2019, result): “We compared our algorithm with other 12 state-ofthe-art deep learning methods, which are LEGS [26], RFCN [28], ELD [13], DCL [16], DS [18], DHS [21], Amulet [33], DSS [10], C2SNet [17], RAS [5], DGRL [29] and PiCANet [22].”
Detect Globally, Refine Locally: A Novel Approach to Saliency Detection (2018, methodology): “We compare the proposed algorithm against 13 stateof-the-art algorithms, including the deep learning based methods as well as other non-deep competitors, DRFI [12], BL [25], LEGS [26], MDF [17], MCDL [36], DS [20], DCL [18], DHS [22], RFCN [28], KSR [30], UCF [34], Amulet [33] and SRM [29].”