Salient Object Detection: A Benchmark

By Ali Borji, Ming-Ming Cheng, Huaizu Jiang, Jia Li. 2015

1,834 citations recorded by the source corpus.

We extensively compare, qualitatively and quantitatively, 41 state-of-the-art models (29 salient object detection, 10 fixation prediction, 1 objectness, and 1 baseline) over seven challenging data sets for the purpose of benchmarking salient object detection and segmentation methods. From the results obtained so far, our evaluation shows a consistent rapid progress over the last few years in terms of both accuracy and running time. The top contenders in this benchmark significantly outperform the models identified as the best in the previous benchmark conducted three years ago. We find that the models designed specifically for salient object detection generally work better than models in closely related areas, which in turn provides a precise definition and suggests an appropriate treatment of this problem that distinguishes it from other problems. In particular, we analyze the influences of center bias and scene complexity in model performance, which, along with the hard cases for the state-of-the-art models, provide useful hints toward constructing more challenging large-scale data sets and better saliency models. Finally, we propose probable solutions for tackling several open problems, such as evaluation scores and data set bias, which also suggest future research directions in the rapidly growing field of salient object detection.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Evaluation & Benchmarks

Citations in the corpus

FocalClick: Towards Practical Interactive Image Segmentation (2022, methodology): “We also report the performance of our model trained on SBD [15], and a large combined dataset [2, 7, 13, 24, 26, 39, 45, 48].”
Res2Net: A New Multi-Scale Backbone Architecture (2019, background): “Early approaches [2] utilize handcrafted representations of global contrast [7] or multi-scale region features [41].”
STC: A Simple to Complex Framework for Weakly-Supervised Semantic Segmentation (2015, background): “DRFI achieves the state-of-the-art performance on six popular benchmark datasets indicated by [19].”
Stacked Cross Refinement Network for Edge-Aware Salient Object Detection (2019, methodology): “More details of them can be found in [2, 3].”
Object Detection With Deep Learning: A Review (2018, methodology): “Among these methods, CHM, RC and DRFI are classical ones with the best performance [159], while the other methods are all associated with CNN.”
Concealed Object Detection (2021, methodology): “We refer readers to recent survey and benchmark papers [11], [34], [35], [36] for more details.”
Hierarchical Image Saliency Detection on Extended CSSD (2016, background): “The goal is to detect and segment out important regions from natural images.”
Enhanced-alignment Measure for Binary Foreground Map Evaluation (2018, background): “, 2014], object detection, recognition [Kanan and Cottrell, 2010; Rutishauser et al. , 2004], foreground extraction [Blake et al. , 2004], and salient object detection [Hou et al. , 2018; Borji et al. , 2015], and is crucial for making statements regarding which models perform better.”
Human Gaze Assisted Artificial Intelligence: A Review (2020, background): “We direct interested readers to recent review papers on the topics of saliency evaluation metrics [Bylinskii et al. , 2019], saliency model performance analyses [Bylinskii et al. , 2016; He et al. , 2019a] and a closely related ﬁeld called salient object detection [Borji et al. , 2015].”
A Stagewise Refinement Model for Detecting Salient Objects in Images (2017, neutral): “, [14, 9, 40, 24, 57, 28, 7].”
Understanding Deep Learning Techniques for Image Segmentation (2019, background): “ding the traditional technologies associated with image segmentation [61, 160]. While some of them specialized in application areas [107, 123, 185], while other focused on specic types of algorithms [20, 19, 59]. With arrival of deep learning techniques many new classes of image segmentation algorit”
Saliency Detection on Light Field (2017, positive): “The saliency detection literature is huge and existing solutions can be classiﬁed in terms of top-down versus bottom-up, center versus background prior, with versus without depth cue, etc. Readers can refer to [5] for a comprehensive comparisons on state-of-the-art solutions.”
Progressive Attention Guided Recurrent Network for Salient Object Detection (2018, neutral): “Detailed introductions of these methods can be found in recent survey paper [1].”
What Makes a Patch Distinct? (2013, methodology): “According to [4], the “Top-4” highest scoring salient object detection algorithms are: SVO [5], CR [6], CNTX [9], and CBS [11].”
Recurrent Attentional Networks for Saliency Detection (2016, methodology): “To have a more comprehensive experimental evaluation, we consider another evaluation metric known as Mean Absolute Error (MAE) adopted by [4].”
SuperCNN: A Superpixelwise Convolutional Neural Network for Salient Object Detection (2015, methodology): “We have compared the proposed method with 7 state-ofthe-art methods, the top three algorithms according to Borji et al. (2012) (RC Cheng et al. 2011, CA Goferman et al. 2010, CBS Jiang et al. 2011), plus four latest algorithms (GMR Yang et al. 2013, HS Yan et al. 2013, PCA Margolin et al. 2013, GC…”
Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks (2019, methodology): “[5], [104], we also provide the max F-measure using various fixed (0-255) thresholds.”
U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection (2020, methodology): “ts against most of the SOTA models (see Fig. 1) at 40 FPS. 2. Related Works In recent years, many deep salient object detection networks [22, 33] have been proposed. Compared with traditional methods [2] based on hand-crafted features like foreground consistency [49], hyperspectral information [20],”
Deep Visual Attention Prediction (2017, methodology): “We refer the reader to two recent literatures: [35] and [36] for more detailed overviews of those two kinds of saliency models.”