The Emergence of Semantics in Neural Network Representations of Visual Information
By Dhanush Dharmaretnam, Alona Fyshe. 2018
6 citations recorded by the source corpus.
Word vector models learn about semantics through corpora. Convolutional Neural Networks (CNNs) can learn about semantics through images. At the most abstract level, some of the information in these models must be shared, as they model the same real-world phenomena. Here we employ techniques previously used to detect semantic representations in the human brain to detect semantic representations in CNNs. We show the accumulation of semantic information in the layers of the CNN, and discover that, for misclassified images, the correct class can be recovered in intermediate layers of a CNN.
Subfields: Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Yonatan Belinkov
- James R. Glass
- R. Thomas McCoy
- Willem H. Zuidema
- Rochelle Choenni
- Samira Abnar
- Michael A. Lepori
- David F. Harwath
In Representation & Self-Supervised Learning
- Yonatan Belinkov
- James R. Glass
- R. Thomas McCoy
- Willem H. Zuidema
- Samira Abnar
- Michael A. Lepori
- Wei-Ning Hsu
- David F. Harwath
Citations in the corpus
- Analysis Methods in Neural Language Processing: A Survey (2018, background): “, 2015) Word embeddings Referential attributes Classification (Dharmaretnam and Fyshe, 2018) Word embeddings, vision CNN Concepts Similarity measures”
- Picking BERT’s Brain: Probing for Linguistic Dependencies in Contextualized Embeddings Using Representational Similarity Analysis (2020, methodology): “Other work has used RSA to compare the semantic representations of CNN object detection models with that of word vectors (Dharmaretnam and Fyshe, 2018), to compare the representations of utterances in a spoken-word encoder to the representations of those same words in the text or”
- Blackbox Meets Blackbox: Representational Similarity & Stability Analysis of Neural Language Models and Brains (2019, background): “RSA can also be applied to deep learning models (Laakso and Cottrell, 2000; Dharmaretnam and Fyshe, 2018; Alvarez-Melis and Jaakkola, 2018; Wang et al., 2018; Chrupała and Alishahi, 2019).”