How Sensitive Is the Human Visual System to the Local Statistics of Natural Images?

By Holly E. Gerhard, Felix Wichmann, Matthias Bethge. 2013

71 citations recorded by the source corpus.

A key hypothesis in sensory system neuroscience is that sensory representations are adapted to the statistical regularities in sensory signals and thereby incorporate knowledge about the outside world. Supporting this hypothesis, several probabilistic models of local natural image regularities have been proposed that reproduce neural response properties. Although many such physiological links have been made, these models have not been linked directly to visual sensitivity. Previous psychophysical studies of sensitivity to natural image regularities focus on global perception of large images, but much less is known about sensitivity to local natural image regularities. We present a new paradigm for controlled psychophysical studies of local natural image regularities and compare how well such models capture perceptually relevant image content. To produce stimuli with precise statistics, we start with a set of patches cut from natural images and alter their content to generate a matched set whose joint statistics are equally likely under a probabilistic natural image model. The task is forced choice to discriminate natural patches from model patches. The results show that human observers can learn to discriminate the higher-order regularities in natural images from those of model samples after very few exposures and that no current model is perfect for patches as small as 5 by 5 pixels or larger. Discrimination performance was accurately predicted by model likelihood, an information theoretic measure of model efficacy, indicating that the visual system possesses a surprisingly detailed knowledge of natural image higher-order correlations, much more so than current image models. We also perform three cue identification experiments to interpret how model features correspond to perceptually relevant image features.

Subfields: Cognitive Neuroscience, Electrical and Electronic Engineering, Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Electrical and Electronic Engineering

In Neuroscience

Citations in the corpus

Are GANs Created Equal? A Large-Scale Study (2017, background): “Unfortunately, such approaches are subjective and possibly misleading [8].”
A note on the evaluation of generative models (2015, methodology): “These issues can be addressed by better image models (e.g., Kingma et al., 2014), better optimization and inference procedures, or a more Bayesian treatment of the parameters (e.g., Lacoste-Julien et al., 2011; Welling & Teh, 2011).”
Pros and Cons of GAN Evaluation Measures (2018, background): “For example, the likelihood is good for measuring compression methods [79] while psychophysics and user ratings are fit for evaluating image reconstruction and synthesis methods [45, 29].”
PsyPhy: A Psychophysics Driven Evaluation Framework for Visual Recognition (2016, background): “[24] introduced a new psychophysical paradigm for comparing human and model sensitivity to local image regularities.”