Learning distributed representations of concepts.

By Geoffrey E. Hinton. 1989

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

TensorFlow: A system for large-scale machine learning (2016, neutral): “To train a model on high-dimensional data, such as words in a corpus of text [7], it is common to use a distributed representation , which embeds a training example as a pattern of activity across several neurons, which can be learned by backpropagation [29].”
Leverage Financial News to Predict Stock Price Movements Using Word Embeddings and Deep Neural Networks (2015, methodology): “[19] provides a easy solution called backpropagation algorithm to address this problem.”
Practical Recommendations for Gradient-Based Training of Deep Architectures (2012, background): “The idea of vector space representations for words and symbols is older (Deerwester et al. , 1990) and is a particular case of the notion of distributed representation (Hinton, 1986, 1989) central to the connectionist approaches.”
Rethinking eliminative connectionism. (1998, methodology): “Sketch of the family tree model of Hinton (1986). Not all units or all connections are shown.”
Hierarchical Probabilistic Neural Network Language Model (2005, background): “The idea of associating each symbol with a distributed continuous representation is not new and was advocated since the early days of neural networks (Hinton, 1986; Elman, 1990).”
Face Model Compression by Distilling Knowledge from Neurons (2016, background): “(1 ) Deeply learned face representation by the face recognition task is a distributed representation [6] over face attributes , including the identity-related attributes (IA), such as gender, race, and shapes of facial components, as well as the identity non-related attributes (NA), such as…”
DeepGO: predicting protein functions from sequence and interactions using a deep ontology-aware classifier (2017, methodology): “To address this limitation, we use the notion of dense embeddings (Hinton, 1986; Bengio et al., 2003).”
Semantic Cognition: A Parallel Distributed Processing Approach (2004, unknown): “When we speak about knowledge of an object’s causal properties, we mean knowledge about how the object participates in events and what the consequences or sequelae are of that participation. This may broaden the concept a bit beyond the uses of others, but certainly still includes what Wilson and Ke”
Continuous space language models (2007, background): “The idea of learning distributed representations for symbolic data was already presented in the early days of connectionism (Hinton, 1986; Elman, 1990) and more recently pursued by Paccanaro and Hinton (2000).”
Survey on categorical data for neural networks (2020, background): “Examples of terms referring to these techniques are “entity embeddings” [3], “dense encoding” [6], “distributed representation” [7], and simply “encoding.”
State-of-the-Art Deep Learning: Evolving Machine Intelligence Toward Tomorrow’s Intelligent Network Traffic Control Systems (2017, background): “For instance, the concept of symbolic data representation through distributed representation was introduced way back in 1986 [79].”