SUN: A Bayesian framework for saliency using natural statistics.

By Lingyun Zhang, Matthew H Tong, Tim K. Marks, Honghao Shan, Garrison W. Cottrell. 2008

1,319 citations recorded by the source corpus.

We propose a definition of saliency by considering what the visual system is trying to optimize when directing attention. The resulting model is a Bayesian framework from which bottom-up saliency emerges naturally as the self-information of visual features, and overall saliency (incorporating top-down information with bottom-up saliency) emerges as the pointwise mutual information between the features and the target when searching for a target. An implementation of our framework demonstrates that our model’s bottom-up saliency maps perform as well as or better than existing algorithms in predicting people’s fixations in free viewing. Unlike existing saliency measures, which depend on the statistics of the particular image being viewed, our measure of saliency is derived from natural image statistics, obtained in advance from a collection of natural images. For this reason, we call our model SUN (Saliency Using Natural statistics). A measure of saliency based on natural image statistics, rather than based on a single test image, provides a straightforward explanation for many search asymmetries observed in humans; the statistics of a single test image lead to predictions that are not consistent with these asymmetries. In our model, saliency is computed locally, which is consistent with the neuroanatomy of the early visual system and results in an efficient algorithm with few free parameters.

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

Citations in the corpus

COCO-Stuff: Thing and Stuff Classes in Context (2016, background): “rform signiﬁcantly above chance at 59.1%, 60.2% and 67.8%, respectively. The relative image position cue performs suprisingly well at 87.7%, conﬁrming the positional bias observed in many prior works [32,52,61,28,4,9]. The more complex objectness measure performs best at 90.6%. Using all cues we ach”
Example Selection For Dictionary Learning (2014, methodology): “This function dictates the way a subset of XN is chosen using GN values.”
Static and space-time visual saliency detection by self-resemblance. (2009, methodology): “[43] fit th e marginal density of local feature vectorsp(f) to a generalized Gaussian distribution.”
Learning to predict where humans look (2009, background): “This bias has often been attributed to the setup of the experiment where users are placed centrally in front of the screen, and to the fact that human photographers tend to place objects of interest in the center of photographs [23].”
Visual search: a retrospective. (2011, methodology): “Since the original proposal, there has been a large number of alternative implementations of the saliency model (Gao & Vasconcelos, 2009; Zhang, Tong, Marks, Shan, & Cottrell, 2008).”
Salient object detection: A survey (2014, background): “s # Model Ref. Pub Year Code 1 IT Itti et al. [1] PAMI 1998 M 2 AIM Bruce & Tsotsos [6] JOV 2006 M 3 GB Harel et al. [164] NIPS 2007 M + C 4 SR Hou & Zhang [18] CVPR 2007 M 5 SUN Zhang et al. [165] JOV 2008 M 6 SeR Seo & Milanfar [61] JOV 2009 M 7 SIM Murray et al. [166] CVPR 2011 M 8 SS”
Saliency Detection: A Boolean Map Approach (2013, methodology): “For example, in [39], it has been shown that a static Gaussian blob has an average ROC score of 0.”
Object Detection in Optical Remote Sensing Images Based on Weakly Supervised Learning and High-Level Feature Learning (2015, neutral): “Although exact maximum likelihood estimation of these parameters is intractable, efficient approximate learning of DBMs can be carried out by using mean-field inference together with the Markov Chain Monte Carlo algorithms [13].”
Revisiting Video Saliency: A Large-Scale Benchmark and a New Model (2018, background): “Early static saliency models [36, 69, 15, 7, 18, 22, 33, 63] are mostly based on the contrast assumption that conspicuous visual features “pop-out” and involuntarily capture attention (see [5, 6] for review).”
Two-Stage Learning to Predict Human Eye Fixations via SDAEs (2016, methodology): “[36] proposed to model the rarity by using a generalized Gaussian model.”