Do We Know What the Early Visual System Does?

By Matteo Carandini, J. Demb, Valerio Mante, David J. Tolhurst, Y. Dan, Barbara Olshausen, Jack L. Gallant, Nicole C. Rust. 2005

681 citations recorded by the source corpus.

We can claim that we know what the visual system does once we can predict neural responses to arbitrary stimuli, including those seen in nature. In the early visual system, models based on one or more linear receptive fields hold promise to achieve this goal as long as the models include nonlinear mechanisms that control responsiveness, based on stimulus context and history, and take into account the nonlinearity of spike generation. These linear and nonlinear mechanisms might be the only essential determinants of the response, or alternatively, there may be additional fundamental determinants yet to be identified. Research is progressing with the goals of defining a single “standard model” for each stage of the visual pathway and testing the predictive power of these models on the responses to movies of natural scenes. These predictive models represent, at a given stage of the visual pathway, a compact description of visual computation. They would be an invaluable guide for understanding the underlying biophysical and anatomical mechanisms and relating neural responses to visual perception.

Subfields: Cellular and Molecular Neuroscience, Cognitive Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cellular and Molecular Neuroscience

In Cognitive Neuroscience

Citations in the corpus

Normalization as a canonical neural computation (2011, background): “It is performed, at least approximately, at various stages in visio”
How does the brain solve visual object recognition? (2012, neutral): “…of a cortical circuit, they are a potentially powerful level of abstraction in that they can account for a substantial amount of single-neuron response patterns in early visual (Carandini et al., 2005), somatosensory (DiCarlo et al., 1998), and auditory cortical areas (Theunissen et al., 2000).”
Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream (2014, background): “Already at the earliest levels of visual processing, there remains ample room for debate as to what form an optimal computational model should take (Carandini et al., 2005).”
Using goal-driven deep learning models to understand sensory cortex (2016, methodology): “Each individual stage of the cascade performs very simple neural operations such as weighted linear sums of inputs or nonlinearities such as activation thresholds and competitive normalization 5 .”
Deep convolutional models improve predictions of macaque V1 responses to natural images (2017, background): “An essential step towards understanding visual processing in the brain is building 2 models that accurately predict neural responses to arbitrary stimuli [1].”
Sparse Modeling for Image and Vision Processing (2014, background): “It is indeed commonly admitted that little is known about the early visual cortex [Olshausen and Field, 2005, Carandini et al., 2005] despite intensive studies and impressive results achieved by predictive models.”
Performance-optimized hierarchical models predict neural responses in higher visual cortex (2014, methodology): “As described in the main text, we used a standard methodology for assessing a model’s ability to predict individual sites (12, 13), in which each site is modeled as a linear combination of model outputs.”
Learning Invariance from Natural Images Inspired by Observations in the Primary Visual Cortex (2012, background): “Optimal stimuli for complex cells do not have to be at a special position in their receptive ﬁelds, that is, complex cells are slightly invariant to position and phase (Hubel & Wiesel, 1962; De Valois, Albrecht, & Thorell, 1982; Adelson & Bergen, 1985; Carandini et al., 2005).”
Deep Neural Networks as a Computational Model for Human Shape Sensitivity (2016, result): “First, we observed that early layers of convnets tended to relate to the physical stimulus dimensions, consistent with the known properties and models of early visual cortex [41].”
How MT cells analyze the motion of visual patterns (2006, background): “The power of L-N models lies in their simplicity, the ease with which they can be fit to data, and their ability to describe stimulus selectivity for a wide variety of neurons (see ref”
Functional mechanisms shaping lateral geniculate responses to artificial and natural stimuli. (2008, background): “Decades of research in retina and LGN have yielded detailed models of the mechanisms shaping responses to simple, artificial stimuli (Carandini et al., 2005; Kaplan and Benardete, 2001; Meister and Berry, 1999; Shapley and Enroth-Cugell, 1984; Troy and Shou, 2002; Victor, 1999).”
Metamers of the ventral stream (2011, background): “, and simple computational models can describe neural responses as a function of visual inpu”
Multistability and Perceptual Inference (2012, background): “The linear-nonlinear Poisson cascade is a canonical building block of neural models, particularly in early vision (Carandini et al., 2005); our model provides a functional interpretation of the cascade in terms of rational perceptual inference.”