3 Learning distributed representations of concepts

By Geoffrey E. Hinton. 2010

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

Neural Turing Machines (2014, background): “In response to this criticism, neural network researchers including Hinton (Hinton, 1986), Smolensky (Smolensky, 1990), Touretzky (Touretzky, 1990), Pollack (Pollack, 1990), Plate (Plate, 2003), and Kanerva (Kanerva, 2009) investigated specific mechanisms that could support both variable-binding and”
DeepWalk: online learning of social representations (2014, background): “Distributed representations have been proposed to model structural relationship between concepts [17].”
Representation Learning: A Review and New Perspectives (2012, methodology): “Distributed representations for symbolic data were introduced by Hinton (1986), and first developed in the context of statistical language modeling by Bengio et al. (2003) in so-called neural net language models (Bengio, 2008).”
A practical tutorial on autoencoders for nonlinear feature fusion: Taxonomy, models, software and guidelines (2018, methodology): “Applying ANNs to learn distributed representations of concepts was proposed by Hinton in [21].”
Active Long Term Memory Networks (2016, background): “t graphical [24, 25] and semantic [26, 27] factors must be stable and slowly evolving to store non-transient knowledge. Early experiments on neural networks’ ability to learn the meso-scale structure [28] of their training environment required interleaved exposure to the different semantic variation”
Crafting adversarial input sequences for recurrent neural networks (2016, methodology): “We convert these integer sequences to matrices, where each row encodes a word as a set of 128 coordinates— known as word embeddings [23], [24].”
Grandmother cells and localist representations: a review of current thinking (2017, methodology): “They trained simple three-layered networks via backpropagation on a variety of tasks, including a logical reasoning task that had previously been simulated by Bechtel and Abrahamsen (1991), and the “kinship problem” problem studied by Hinton (1986).”
Harvesting Common-sense Navigational Knowledge for Robotics from Uncurated Text Corpora (2017, result): “They are not the first to perform such a feat, but their machine learning approach was more flexible than the statistical methods [9, 10] that preceded them and more efficient than similar approaches which used more complex network architectures [11, 5, 12].”
The Application of Deep Learning in Cancer Prognosis Prediction (2020, methodology): “descent algorithm during backpropagation to minimize global loss function [43].”
Discovering Internal Representations from Object-CNNs Using Population Encoding (2015, methodology): “CNNs have been especially successful at many difficult computer vision tasks such as image classification, as showcased by Krizhevsky et al. (2012) and Simonyan & Zisserman (2015), at object detection as shown by Girshick et al.”
Statistically Significant Detection of Linguistic Change (2014, background): “arge corpora. Moreover, our work is unique in the fact that our datasets span dierent time scales, cover larger user interactions and represent a better sample of the web. 7.2 Word Embeddings Hinton [19] proposed distributed representations (word embeddings), to learn a mapping of symbolic data to c”
On the Expressive Power of Deep Architectures (2011, background): “Distributed representations were put forward in the early days of connectionism and artificial neural networks (Hinton, 1986, 1989).”
Using matrices to model symbolic relationship (2008, background): “The “standard” family trees task defined in [3] consists of the two family trees shown in figure 1(a) where the relations are {has husband, has wife, has son, has daughter, has father, has mother, has brother, has sister, has nephew, has niece, has uncle, has aunt}.”
Toward Deep Learning Software Repositories (2015, background): “These neural activation patterns, or distributed representations [36], harness formidable and efficient internal representations of domain concepts.”
A Survey on Context Learning (2017, background): “The distributed representation was first proposed by [46] and introduced into natural language processing by [2].”
Sentiment Analysis of Comment Texts Based on BiLSTM (2019, background): “Distributed representation was proposed by Hinton in the 1986 [27].”