Visual saliency estimation by nonlinearly integrating features using region covariances.

By Erkut Erdem, Aykut Erdem. Journal of Vision, 2013

416 citations recorded by the source corpus.

To detect visually salient elements of complex natural scenes, computational bottom-up saliency models commonly examine several feature channels such as color and orientation in parallel. They compute a separate feature map for each channel and then linearly combine these maps to produce a master saliency map. However, only a few studies have investigated how different feature dimensions contribute to the overall visual saliency. We address this integration issue and propose to use covariance matrices of simple image features (known as region covariance descriptors in the computer vision community; Tuzel, Porikli, & Meer, 2006) as meta-features for saliency estimation. As low-dimensional representations of image patches, region covariances capture local image structures better than standard linear filters, but more importantly, they naturally provide nonlinear integration of different features by modeling their correlations. We also show that first-order statistics of features could be easily incorporated to the proposed approach to improve the performance. Our experimental evaluation on several benchmark data sets demonstrate that the proposed approach outperforms the state-of-art models on various tasks including prediction of human eye fixations, salient object detection, and image-retargeting.

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition, Sensory Systems

View Paper: https://www.semanticscholar.org/paper/9896b03aeb17e06181c1842773fec4b742d7c51f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

In Sensory Systems

Citations in the corpus

Salient object detection: A survey (2014, methodology): “s can handle both ﬁxation prediction and salient object detection (e.g., BMS [110]). Some authors have thresholded their saliency maps (e.g., top 20% of activation) to detect proto-objects (e.g., COV [62] models). Fig.7provides a list of ﬁxation prediction models considered in this study. All of the”
Scene-Level Geographic Image Classification Based on a Covariance Descriptor Using Supervised Collaborative Kernel Coding (2016, neutral): “, texture discrimination [18], visual saliency estimation [19], object detection [18,20] and object tracking [21].”
SALICON: Reducing the Semantic Gap in Saliency Prediction by Adapting Deep Neural Networks (2015, methodology): “We compare our results with 14 saliency prediction models, namely BMS [42], AWS [11], AIM [2], RARE [30], LG [1], eDN [39], Judd [21], SigSal [15], CAS [12], CovSal [8], ΔQDCT [32], SUN [43], GBVS [14], and ITTI [16].”
Salient Object Detection: A Benchmark (2015, background): “Since there exist many datasets that differ in number of images, number of objects per image, image resolution and annotation form (bounding box or accurate region mask), it is likely that models may rank differently across datasets.”
Attentive Systems: A Survey (2017, background): “…maximization (AIM, Bruce et al. 2005), c boolean map based saliency (BMS, Zhang and Sclaroff 2013), d saliency based on region covariance (COV, Erdem and Erdem 2013), e graph based saliency (GB, Harel et al. 2006), f incremental coding length (ICL, Hou and Zhang 2008), g visual attention…”
Large-Scale Optimization of Hierarchical Features for Saliency Prediction in Natural Images (2014, methodology): “Our model, Ensemble of Deep Networks (eDN), comes closest to human data on this benchmark; also shown are Judd and CovSal (ranked 2nd and 3rd, respectively). tense research both in neuroscience [15] and in computer vision, where saliency-based preprocessing has found a wide applicability from image/”
ConceptMap: Mining Noisy Web Data for Concept Learning (2014, background): “Learning Object Categories: In the case of objects, we detect salient regions on each image via [11], to eliminate background noise.”