Visual Saliency Based on Scale-Space Analysis in the Frequency Domain

By Jian Li, Martin David Levine, Xiangjing An, Xin Xu, Hangen He. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013

587 citations recorded by the source corpus.

We address the issue of visual saliency from three perspectives. First, we consider saliency detection as a frequency domain analysis problem. Second, we achieve this by employing the concept of nonsaliency. Third, we simultaneously consider the detection of salient regions of different size. The paper proposes a new bottom-up paradigm for detecting visual saliency, characterized by a scale-space analysis of the amplitude spectrum of natural images. We show that the convolution of the image amplitude spectrum with a low-pass Gaussian kernel of an appropriate scale is equivalent to an image saliency detector. The saliency map is obtained by reconstructing the 2D signal using the original phase and the amplitude spectrum, filtered at a scale selected by minimizing saliency map entropy. A Hypercomplex Fourier Transform performs the analysis in the frequency domain. Using available databases, we demonstrate experimentally that the proposed model can predict human fixation data. We also introduce a new image database and use it to show that the saliency detector can highlight both small and large salient regions, as well as inhibit repeated distractors in cluttered images. In addition, we show that it is able to predict salient regions on which people focus their attention.

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/0c760f678c99b4a7eacc4b0ef7a42d4e6c734983

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

Citations in the corpus

Spatiotemporal Knowledge Distillation for Efficient Estimation of Aerial Video Saliency (2019, methodology): “1) The Heuristic Group (denoted as H Group) contains three heuristic models, including HFT [50], SP [51] and PNSP [25].”
Visual saliency estimation by nonlinearly integrating features using region covariances. (2013, methodology): “In Li et al. (2012), the AUC score and the maximal value of the Dice Similarity Coefficient (DSC) curve were used in quantitative evaluation.”
Saliency Detection: A Boolean Map Approach (2013, methodology): “Although several compared methods have similar performance as BMS on some of the datasets, e.g. AWS [10] on MIT and Kootstra, ΔQDCT [33] on Toronto and Cerf, GBVS [13] on ImgSal, none of them achieves top performance on all five datasets.”
Two-Stage Learning to Predict Human Eye Fixations via SDAEs (2016, methodology): “[38] performed the convolution of the image amplitude spectrum with a low-pass Gaussian kernel of an appropriate scale to detect saliency.”
What Do Different Evaluation Metrics Tell Us About Saliency Models? (2016, methodology): “[68] provided an evaluation 12 saliency models with 12 similarity metrics on Jian Li’s dataset [48].”
Contextual Hypergraph Modeling for Salient Object Detection (2013, background): “Imgsal-50 is a subset of the Imgsal dataset [31], and comprises 50 images with large salient objects for evaluation.”
Saliency and Human Fixations: State-of-the-Art and Study of Comparison Metrics (2013, methodology): “HFT is also an efficient model which is proposed by the database author Jian Li [18].”
Saliency Weighted Features for Person Re-identification (2014, background): “cameras we have to deal with background clutter that may induce state-ofthe-art saliency detection algorithms [22, 3, 47, 21, 44, 15, 26] to label “salient” a background region.”
The Secrets of Salient Object Segmentation (2014, methodology): “In this paper, we will benchmark on the following datasets: Bruce [6], Judd [18], Cerf [8], FT [1], and IS [20].”
New Contour Cue-Based Hybrid Sparse Learning for Salient Object Detection (2019, unknown): “…deeper insights into the performance of the three distance metrics, we build a veriﬁcation set (with images collected from the dataset of Li et al. [69]), which contains scenes with salient objects of variable sizes or cropped at the border, complex fore/back-ground areas and low region contrast.”
Salient Object Detection via Structured Matrix Decomposition (2017, background): “the theories by taking account of local [43], [44], regional [45], and global [46] contrast cues, or by searching for saliency cues in the frequency domain [14], [47].”
Contextual Hypergraph Modelling for Salient Object Detection (2013, background): “Imgsal-50 is a subset of the Imgsal dataset [31], and comprises 50 images with large salient objects for evaluation.”