Metamers of the ventral stream

By Jeremy Freeman, Eero P. Simoncelli. 2011

715 citations recorded by the source corpus.

The human capacity to recognize complex visual patterns emerges in a sequence of brain areas known as the ventral stream, beginning with primary visual cortex (V1). We developed a population model for mid-ventral processing, in which nonlinear combinations of V1 responses are averaged in receptive fields that grow with eccentricity. To test the model, we generated novel forms of visual metamers, stimuli that differ physically but look the same. We developed a behavioral protocol that uses metameric stimuli to estimate the receptive field sizes in which the model features are represented. Because receptive field sizes change along the ventral stream, our behavioral results can identify the visual area corresponding to the representation. Measurements in human observers implicate visual area V2, providing a new functional account of neurons in this area. The model also explains deficits of peripheral vision known as crowding, and provides a quantitative framework for assessing the capabilities and limitations of everyday vision.

Subfields: Cognitive Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

Citations in the corpus

Excessive Invariance Causes Adversarial Vulnerability (2018, background): “Metamerism refers to the cognitive equivalent to the preimage in neuroscience (Wandell, 1995; Freeman & Simoncelli, 2011; Wallis et al., 2016).”
Exploring the structure of a real-time, arbitrary neural artistic stylization network (2017, background): “Images with identical lower-order spatial statistics appear perceptually identical and capture a visual texture [6, 8, 21, 26].”
Texture synthesis and the controlled generation of natural stimuli using convolutional neural networks (2015, background): “In particular the parametric texture model proposed by Portilla and Simoncelli [15] has sparked a great number of studies in neuroscience and psychophysics [5, 4, 1, 16, 14].”
Learning Invariant Feature Hierarchies (2012, background): “Units have local receptive fields whose size increases as we move up in the hierarchy [13].”
Neural networks and neuroscience-inspired computer vision. (2014, neutral): “While the original image and the altered image are classiﬁed as completely different objects by the deep learning system, they are effectively indistinguishable by humans (they would be considered to be ‘metamers’ in the language of visual psychophysics [52]).”
Nonlinear Hebbian Learning as a Unifying Principle in Receptive Field Formation (2016, neutral): “d dt w(t) = Ay(t)ȳ(t)x̄(t)− A−x(t)ȳ−(t) (4)”
Computational role of eccentricity dependent cortical magnification (2014, background): “4 (see also [6]), our theory predicts that an interference-free signature from V2 is important for recognition.”
Performance-optimized hierarchical models predict neural responses in higher visual cortex (2014, methodology): “This parameter space expresses an inclusive version of the hierarhical feedforward network concept and contains models similar to that used in many previous studies for different parameter values (19, 23, 24, 25).”
Texture Synthesis Using Convolutional Neural Networks (2015, background): “In particular the parametric texture model proposed by Portilla and Simoncelli [21] has sparked a great number of studies in neuroscience and psychophysics [8, 7, 1, 22, 20].”
Neuroevolution of self-interpretable agents (2020, background): “anatomical and physiological studies inmonkeys suggest that visual signals are fed into processing systems to extract high level concepts such as shape, color and spatial organization [1, 28].”
A Completely Blind Video Integrity Oracle (2016, methodology): “in [43] of computing space-time correlations (used successfully there to model visual crowding) as a simple measure of”
How Sensitive Is the Human Visual System to the Local Statistics of Natural Images? (2013, methodology): “Psychophysical studies using the Portilla-Simoncelli model have advanced our understanding of peripheral visual processing [54,55], and have uncovered physiological properties of early visual cortex [56].”
Unsupervised Learning of Invariant Representations in Hierarchical Architectures (2013, background): “Another HMAX variant added a time dimension for action recognition [20], outperforming both human annotators and a state-of-the-art commercial system on a mouse behavioral phenotyping task.”
Unsupervised neural network models of the ventral visual stream (2020, background): “Midlevel visual areas such as V2, V3, and V4 are less well understood, but appear to perform intermediate computations between simple edges and complex objects, correlating with sequentially increasing receptive ﬁeld size (6–14).”
Artificial Neural Networks for Neuroscientists: A Primer (2020, background): “…comparison, classical models of the visual systems typically rely on hand-designed features (synaptic weights) (Jones and Palmer, 1987; Freeman and Simoncelli, 2011; Riesenhuber and Poggio, 1999), such as Gabor filters, or are trained with unsupervised learning based on the efficient coding…”