How Deep is the Feature Analysis underlying Rapid Visual Categorization?

By Sven Eberhardt, Jonah Cader, Thomas Serre. 2016

53 citations recorded by the source corpus.

Rapid categorization paradigms have a long history in experimental psychology: Characterized by short presentation times and speedy behavioral responses, these tasks highlight the efficiency with which our visual system processes natural object categories. Previous studies have shown that feed-forward hierarchical models of the visual cortex provide a good fit to human visual decisions. At the same time, recent work in computer vision has demonstrated significant gains in object recognition accuracy with increasingly deep hierarchical architectures. But it is unclear how well these models account for human visual decisions and what they may reveal about the underlying brain processes. We have conducted a large-scale psychophysics study to assess the correlation between computational models and human participants on a rapid animal vs. non-animal categorization task. We considered visual representations of varying complexity by analyzing the output of different stages of processing in three state-of-the-art deep networks. We found that recognition accuracy increases with higher stages of visual processing (higher level stages indeed outperforming human participants on the same task) but that human decisions agree best with predictions from intermediate stages. Overall, these results suggest that human participants may rely on visual features of intermediate complexity and that the complexity of visual representations afforded by modern deep network models may exceed those used by human participants during rapid categorization.

Subfields: Cognitive Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

Citations in the corpus

Global-and-local attention networks for visual recognition (2018, methodology): “While recent advancements in DCNs have led to models that perform on par with human observers in basic visual recognition tasks, there is also growing evidence of qualitative differences in the visual strategies that they employ [47, 48, 39, 35].”
What are the Visual Features Underlying Human Versus Machine Vision? (2017, background): “It has been suggested that the processing depth achieved by state-of-the-art DCNs may be greater than that achieved by the human visual system during rapid categorization [7].”
PsyPhy: A Psychophysics Driven Evaluation Framework for Visual Recognition (2016, methodology): “[22] designed an experiment to measure human accuracy and reaction time during visual categorization tasks with natural images, which were then compared to different layers of CNNmodels [6], [19].”
The Notorious Difficulty of Comparing Human and Machine Perception (2020, background): “…prediction or internal representations are compared to those of biological systems; for example to human and/or monkey behavioral representations (Eberhardt, Cader, & Serre, 2016; Golan, Raju, & Kriegeskorte, 2019; Peterson, Abbott, & Griﬃths, 2016; Schrimpf et al., 2018; Yamins et al., 2014),…”