Learning multiple layers of representation.

By Geoffrey E. Hinton. 2007

1,087 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Representation & Self-Supervised Learning, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Representation & Self-Supervised Learning

In Signal Processing

Citations in the corpus

Deep neural networks are easily fooled: High confidence predictions for unrecognizable images (2014, background): “Deep neural networks (DNNs) learn hierarchical layers of representation from sensory input in order to perform pattern recognition [2, 14].”
NIPS 2016 Tutorial: Generative Adversarial Networks (2016, methodology): “Boltzmann machines were an important part of the deep learning renaissance beginning in 2006 (Hinton et al., 2006; Hinton, 2007) but they are now used only very rarely, presumably mostly because the underlying Markov chain approximation techniques have not scaled to problems like ImageNet generation”
Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks (2015, background): “These layers can be seen as successive representations of the input data [23], a multidimensional vector X , each of them corresponding to one of the parametric functions mentioned above.”
Giraffe: Using Deep Reinforcement Learning to Play Chess (2015, background): “A new class of algorithms capable of performing hierarchical knowledge extraction were popularized in 2007 by Hinton et al.[22].”
Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning (2018, background): “This is paradoxical because of the very nature of deep learning: an essential part of the design philosophy of DNNs is to learn a modular model whose components (layers of neurons) are simple in isolation yet powerful and expressive in combination—thanks to their orchestration as a composition of no”
The global landscape of cognition: hierarchical aggregation as an organizational principle of human cortical networks and functions (2015, methodology): “(1) BrainMap is a database of manually entered fMRI publications including activation coordinates associated with tasks, among other meta-data categories21–24.”
Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA (2016, methodology): “Some of the most popular methods are multi-layer belief nets and Restricted Boltzmann Machines [13] as well as autoencoders [14, 31, 21], which form the basis for the ladder networks [30].”
Deep Metric Learning Using Triplet Network (2014, background): “One of the underlying assumptions is that deep, hierarchical models such as convolutional networks create useful representation of data (Bengio (2009); Hinton (2007)), which can then be used to distinguish between available classes.”
Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream (2014, background): “From a computational point of view it is not inconceivable that unsupervised or reinforcement learning schemes, which allow learning of multiple layers of increasingly complex stimulus features (Hinton, 2007; Mnih et al., 2015), will outperform DNN-based encoding models in explaining neural response”
Importance Sampling for Minibatches (2016, background): “…ever increasing demand for accuracy of the predictors, largely due to successful industrial applications, and with ever more sophisticated models that need to trained, such as deep neural networks [8, 14], or multiclass classiﬁcation [9], increasing volumes of data are used in the training phase.”
AgeDB: The First Manually Collected, In-the-Wild Age Database (2017, methodology): “Before the advent of deep learning in 2006 [16], construction of algorithms that were used in facial analysis tasks required huge amount of time, domain specific knowledge and a delicate engineering process in order to transform the raw facial data in a feature space.”
Marketing Analytics for Data-Rich Environments (2016, neutral): “Deep neural networks (Hinton 2007) provide good prediction results for voice recognition, natural language processing, visual recognition and classi ﬁ cation (especially objects and scenes in images and video), and computer game playing.”
Your evidence? Machine learning algorithms for medical diagnosis and prediction (2019, background): “Advances in both algorithms (Hinton, 2007) and hardware (GPUs and subsequently TPUs) have resulted in much more efficient and powerful estimation of deep architectures, and have subsequently given rise to a distinction featuring prominently in current debates, namely the one between “ conventional…”
Unsupervised Deep Learning Applied to Breast Density Segmentation and Mammographic Risk Scoring (2016, positive): “The features can also be learned in an unsupervised way, e.g., using Restricted Boltzmann Machines [48], [49] or autoen-coders [23], [50], [51].”
A tutorial on task-parameterized movement learning and retrieval (2015, background): “This can be exploited as a link with deep learning strategies [40, 9] for real-valued high-dimensional data within directed graphical models.”