Distributed Representations

By Geoffrey E. Hinton, James L. McClelland, David E. Rumelhart. 1986

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

Experience Grounds Language (2020, background): “…tension of modeling symbols and distributed representations was nicely articulated by Smolensky (1990), and alternative representations (Kohonen, 1984; Hinton et al., 1986; Barlow, 1989) and approaches to structure and composition (Erk and Padó, 2008; Socher et al., 2012) span decades of research.”
Efficient Estimation of Word Representations in Vector Space (2013, background): “Probably the most successful concept is to use distributed representations of words [10].”
Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks (2018, background): “The metric is of interest to our work because it provides some measure of the degree to which features are being shared across classes , a central property of distributed representations that can describe concepts efﬁciently [12].”
On the Binding Problem in Artificial Neural Networks (2020, background): “But unlike symbols, they retain the expressive distributed feature-based internal structure of connectionist representations, which are known to facilitate generalization (Hinton, 1984; Bengio et al., 2013).”
Generalization Through the Recurrent Interaction of Episodic Memories (2012, background): “…and distributed in nature, with any single neuron participating in the representation of more than one entity or episode (Hinton, McClelland, & Rumelhart, 1986; McClelland et al., 1995; Plaut & McClelland, 2010; Quiroga, Kreiman, Koch, & Fried, 2008; Waydo, Kraskov, Quiroga, Fried, & Koch,…”
Neural Expectation Maximization (2017, background): “It has been argued that such representations should be distributed [13, 37] and disentangled [1, 31, 3].”
Continuous Distributed Representation of Biological Sequences for Deep Proteomics and Genomics (2015, neutral): “Distributed representation has proved one of the most successful approaches in machine learning [6 – 10].”
A Disentangling Invertible Interpretation Network for Explaining Latent Representations (2020, background): “ributed coding is crucial for the robustness and generalization performance of neural networks despite noisy inputs, large intra-class variability, and the stochastic nature of the learning algorithm [13]. However, as a downside of semantics being distributed over numerous neurons it is impossible t”
A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages (2020, background): “The most notable examples are word2vec (Mikolov et al., 2013), GloVe (Pennington et al., 2014) and fastText (Mikolov et al., 2018).”
Inductive biases for deep learning of higher-level cognition (2020, background): “ible to generalize from a ﬁnite set of combinations to a larger set of combinations of concepts. Deep learning already beneﬁts from a form of compositional advantage with distributed representations (Hinton, 1984; Bengio and Bengio, 2000; Bengio et al., 2001), which are at the heart of why neural ne”
From Dependence to Causation (2016, background): “One key property fueling the power of deep representations is that these are distributed representations (Hinton et al., 1986).”
Imitative Reinforcement Learning for Soccer Playing Robots (2006, methodology): “We construct the features following the principles of coarse coding [11].”
Knowledge-Powered Deep Learning for Word Embedding (2014, background): “Obtaining continious word embedding has been studied for a long time [9].”