Selectivity and tolerance (“invariance”) both increase as visual information propagates from cortical area V4 to IT

By Nicole C. Rust, James J. DiCarlo. Journal of Neuroscience, 2010

393 citations recorded by the source corpus.

Our ability to recognize objects despite large changes in position, size, and context is achieved through computations that are thought to increase both the shape selectivity and the tolerance (“invariance”) of the visual representation at successive stages of the ventral pathway [visual cortical areas V1, V2, and V4 and inferior temporal cortex (IT)]. However, these ideas have proven difficult to test. Here, we consider how well population activity patterns at two stages of the ventral stream (V4 and IT) discriminate between, and generalize across, different images. We found that both V4 and IT encode natural images with similar fidelity, whereas the IT population is much more sensitive to controlled, statistical scrambling of those images. Scrambling sensitivity was proportional to receptive field (RF) size in both V4 and IT, suggesting that, on average, the number of visual feature conjunctions implemented by a V4 or IT neuron is directly related to its RF size. We also found that the IT population could better discriminate between objects across changes in position, scale, and context, thus directly demonstrating a V4-to-IT gain in tolerance. This tolerance gain could be accounted for by both a decrease in single-unit sensitivity to identity-preserving transformations (e.g., an increase in RF size) and an increase in the maintenance of rank-order object selectivity within the RF. These results demonstrate that, as visual information travels from V4 to IT, the population representation is reformatted to become more selective for feature conjunctions and more tolerant to identity preserving transformations, and they reveal the single-unit response properties that underlie that reformatting.

Subfields: Cognitive Neuroscience

View Paper: https://www.semanticscholar.org/paper/0e0f746ceadfea572c0850c120b5a1388f98d458

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

Citations in the corpus

Invariant object recognition is a personalized selection of invariant features in humans, not simply explained by hierarchical feed-forward vision models (2017, background): “More specifically, humans used a set of diagnostic features which were more consistent and less numerous across variation conditions on the LSD (Figs 9 and 10).”
The Limits of Feedforward Vision: Recurrent Processing Promotes Robust Object Recognition when Objects Are Degraded (2012, background): “…has been reported to impair object recognition (Grill-Spector et al., 2000; Rolls et al., 1999), receptive fields subtend large portions of the visual field (generally 10–20°; Rust & Dicarlo, 2010; Kobatake & Tanaka, 1994), and thus, spatial-based inhibition at this level is virtually nonexistent.”
A category-free neural population supports evolving demands during decision-making (2014, methodology): “We assembled as many pseudo-trials as possible by randomly sampling trials from each neuron without replacement; because the number of trials from each neuron was balanced across conditions when training the classifiers, the process was limited by the neuron with the fewest trials in any one conditi”
Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition (2014, methodology): “Results using linear classifiers have shown that the IT neural representation creates a simpler object recognition problem that can often be solved with a linear function predictive of object category [9,10].”
Look and Think Twice: Capturing Top-Down Visual Attention with Feedback Convolutional Neural Networks (2015, background): “Despite great successes achieved by applying Feedforward Networks to image recognition and detection, evidences accumulate from cognitive studies and point to the feedback mechanism that may dominant human perception processes [3, 22, 16, 19].”
Deep Supervised, but Not Unsupervised, Models May Explain IT Cortical Representation (2014, background): “However, the IT representation has been shown to be less sensitive to changes of position, size, and context than earlier stages of processing [79].”
Figure–ground organization and the emergence of proto-objects in the visual cortex (2015, background): “Examining the ventral stream from V1 to anterior IT cortex, one finds increasing size of receptive fields, increasing shape selectivity and increasing position tolerance (Zeki, 1978; Desimone et al., 1985; Tanaka et al., 1991; Connor et al., 2007; Rust and DiCarlo, 2010).”
Correspondence of Deep Neural Networks and the Brain for Visual Textures (2018, neutral): “An increase in error might be expected at higher layers, similar to the observation that area V4 better discriminates synthesized “jumbled image” textures than area IT [25], [42].”
THINGS: A database of 1,854 object concepts and more than 26,000 naturalistic object images (2019, background): “…et al., 2008) to object renderings placed on naturalistic scenes (e.g. Yamins et al., 2014), object images with a naturalistic background (e.g. Rust & DiCarlo, 2010; Thorpe, Fize, & Marlot, 1996), multiple objects in naturalistic scenes (e.g. Peelen, Fei-Fei, & Kastner, 2009; Torralba, Oliva,…”