Adapting Deep Network Features to Capture Psychological Representations

By Joshua C. Peterson, Joshua T. Abbott, T. Griffiths. 2016

72 citations recorded by the source corpus.

Deep neural networks have become increasingly successful at solving classic perception problems such as object recognition, semantic segmentation, and scene understanding, often reaching or surpassing human-level accuracy. This success is due in part to the ability of DNNs to learn useful representations of high-dimensional inputs, a problem that humans must also solve. We examine the relationship between the representations learned by these networks and human psychological representations recovered from similarity judgments. We find that deep features learned in service of object classification account for a significant amount of the variance in human similarity judgments for a set of animal images. However, these features do not capture some qualitative distinctions that are a key part of human representations. To remedy this, we develop a method for adapting deep features to align with human similarity judgments, resulting in image representations that can potentially be used to extend the scope of psychological experiments.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

Citations in the corpus

Large-Scale, High-Resolution Comparison of the Core Visual Object Recognition Behavior of Humans, Monkeys, and State-of-the-Art Deep Artificial Neural Networks (2018, background): “To this end, previous work had shown that specific ANNs drawn from a large family of DCNNs and optimized to achieve high levels of object categorization performance on large-scale image sets capture a large fraction of the variance in primate visual recognition behaviors (Ghodrati et al., 2014; Raja”
Decoding the Deep: Exploring Class Hierarchies of Deep Representations Using Multiresolution Matrix Factorization (2017, background): “Using classical techniques may not easily reveal interesting, human-relateable trends as recently shown by [26].”
What are the Visual Features Underlying Human Versus Machine Vision? (2017, methodology): “Recent work has used this approach to compare representations between humans and representative DCNs, finding good agreement between the two [21].”
The Notorious Difficulty of Comparing Human and Machine Perception (2020, unknown): “…systems; for example to human and/or monkey behavioral representations (Eberhardt, Cader, & Serre, 2016; Golan, Raju, & Kriegeskorte, 2019; Peterson, Abbott, & Griﬃths, 2016; Schrimpf et al., 2018; Yamins et al., 2014), human fMRI representations (Han et al., 2019; Khaligh-Razavi &…”