Amulet: Aggregating Multi-level Convolutional Features for Salient Object Detection

By Pingping Zhang, Dong Wang, Huchuan Lu, Hongyu Wang, Xiang Ruan. IEEE International Conference on Computer Vision, 2017

790 citations recorded by the source corpus.

Fully convolutional neural networks (FCNs) have shown outstanding performance in many dense labeling problems. One key pillar of these successes is mining relevant information from features in convolutional layers. However, how to better aggregate multi-level convolutional feature maps for salient object detection is underexplored. In this work, we present Amulet, a generic aggregating multi-level convolutional feature framework for salient object detection. Our framework first integrates multi-level feature maps into multiple resolutions, which simultaneously incorporate coarse semantics and fine details. Then it adaptively learns to combine these feature maps at each resolution and predict saliency maps with the combined features. Finally, the predicted results are efficiently fused to generate the final saliency map. In addition, to achieve accurate boundary inference and semantic enhancement, edge-aware feature maps in low-level layers and the predicted results of low resolution features are recursively embedded into the learning framework. By aggregating multi-level convolutional features in this efficient and flexible manner, the proposed saliency model provides accurate salient object labeling. Comprehensive experiments demonstrate that our method performs favorably against state-of-the-art approaches in terms of near all compared evaluation metrics.

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/8410ee05f62b4f5d295917d2f56e0547ad5edb07

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

Citations in the corpus

RGB-D salient object detection: A survey (2020, background): “Although significant progress has made in the SOD field over the past several years [50, 73, 73, 77, 119, 136, 142, 150, 151, 157, 168, 169, 174, 176], there is still room for improvement under challenging factors such as complicated background or different lighting conditions”
Attentive Feedback Network for Boundary-Aware Salient Object Detection (2019, background): “We compared our algorithm with other 12 state-ofthe-art deep learning methods, which are LEGS [26], RFCN [28], ELD [13], DCL [16], DS [18], DHS [21], Amulet [33], DSS [10], C2SNet [17], RAS [5], DGRL [29] and PiCANet [22].”
Interactive Two-Stream Decoder for Accurate and Fast Saliency Detection (2020, methodology): “F-score and FPS comparisons of our ITSD with AFNet [8], PCA [22], Amulet [45], EGNet [48], CKT [18], CTLoss [4], NLDF [24], PoolNet [21], PAGE [35] and CPD [37] on DUTS-TE dataset [31].”
Learning Multi-Level Density Maps for Crowd Counting (2020, background): “Amulet [44] ﬁrst generates multi-level features and then integrates them into multiple resolutions by adaptively learning the corresponding combinations.”
A Bi-Directional Message Passing Model for Salient Object Detection (2018, methodology): “This motivates several efforts [37, 8, 14] to exploit multi-level convolutional features for saliency detection.”
Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks (2019, methodology): “[5], [104], we also provide the max F-measure using various fixed (0-255) thresholds.”
The impact of pre- and post-image processing techniques on deep learning frameworks: A comprehensive review for digital pathology image analysis (2020, background): “The decoder network aims to semantically project the discriminating characteristics (lower resolution) learned by the encoder on the pixel space (higher resolution) to obtain a dense classification [176].”
BBS-Net: RGB-D Salient Object Detection with a Bifurcated Backbone Strategy Network (2020, methodology): “ semantic information. Recently, to solve this problem, deep learning based methods [47]–[51] have been explored, exceeding handcrafted features based methods in complex scenarios. These deep methods [52] usually leverage CNNs to extract multilevel multi-scale features from RGB images and then aggre”
Pyramid Dilated Deeper ConvLSTM for Video Salient Object Detection (2018, methodology): “, multi-stream network with embedded superpixels [22, 25], recurrent module [26, 40], and multi-scale and hierarchical feature fusion [16, 51, 36], etc.”
U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection (2020, background): “(Amulet) [53] predict saliency maps by aggregating multi-level features into different resolutions.”
How Much Position Information Do Convolutional Neural Networks Encode? (2020, methodology): “Following the common training protocol used in (Zhang et al., 2017; Liu et al., 2018), we train the model on the training set of DUT-S and evaluate the existence of position information on the natural images of the PASCAL-S (Li et al., 2014) dataset.”
Surface defect saliency of magnetic tile (2018, methodology): “Encoder-decoder structures generated from Fully Convolutional Network(FCN) [22] are mostly used in cuttingedge saliency detection models [17], [23], [24].”
Detect Globally, Refine Locally: A Novel Approach to Saliency Detection (2018, methodology): “Existing saliency detection methods [18, 10, 33] attempt to combine hierarchical features to capture distinctive objectness and detailed information simultaneously.”
Label Decoupling Framework for Salient Object Detection (2020, methodology): “In addition to saliency masks, edge label is also introduced to SOD in [23, 34, 31, 20, 39, 42] to assist the generation of saliency maps.”
EGNet: Edge Guidance Network for Salient Object Detection (2019, methodology): “Image GT Ours PiCANet [33] PAGR [63] DGRL [52] SRM [49] UCF [62] Amulet [61] DSS [17, 18] DHS [32]”
A Simple Pooling-Based Design for Real-Time Salient Object Detection (2019, methodology): “When the edge branch is not incorporated, training only takes less than 6 hours on a training set of 5,000 images, which is quite faster than most of the previous methods [24, 43, 28, 44, 45, 9].”
BASNet: Boundary-Aware Salient Object Detection (2019, methodology): “Zhang et al. (UCF) [75] developed a reformulated dropout and a hybrid upsampling module to reduce the checkboard artifacts of deconvolution operators as well as aggregating multi-level convolutional features in (Amulet) [74] for saliency detection.”