To recognize shapes, first learn to generate images.

By Geoffrey E. Hinton. 2007

346 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Biophysics, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biophysics

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Distributed Prioritized Experience Replay (2018, background): “This has been shown to be useful in the context of neural networks (Hinton, 2007).”
Deep, Big, Simple Neural Nets for Handwritten Digit Recognition (2010, background): “Training them is hard as back-propagated gradients quickly vanish exponentially in the number of layers (Hochreiter, 1991; Hochreiter et al., 2001; Hinton, 2007), just like in the first recurrent neural networks (Hochreiter & Schmidhuber, 1997).”
Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samples (2017, methodology): “Selecting more difficult instances can accelerate learning or improve testing accuracy [18, 29, 13].”
A Deep Non-linear Feature Mapping for Large-Margin kNN Classification (2009, result): “Our findings verified the hypothesis discussed in [10].”
3D Object Recognition with Deep Belief Nets (2009, methodology): “In this paper, we describe a better type of top-level model for deep belief nets that is trained using a combination of generative and discriminative gradients [5], [8], [9].”
Large-scale kernel machines (2007, neutral): “Inspired by [Hinton, To appear.”
An empirical evaluation of deep architectures on problems with many factors of variation (2007, background): “Hinton (2006) gives empirical evidence that Appearing in Proceedings of the 24 th International Conference on Machine Learning, Corvallis, OR, 2007.”
Network anomaly detection with the restricted Boltzmann machine (2013, background): “Deep Belief Networks [15,14] are obtained by stacking RBMs on one another so that the input to one layer is given by the hidden units of the adjacent layer, as if they were data, and adding a last discriminative layer.”
Discovering Binary Codes for Documents by Learning Deep Generative Models (2011, methodology): “The model can be trained efﬁciently by using an RBM to learn one layer of hidden variables at a time (Hinton et al., 2006; Hinton, 2007a).”
Why does Deep Learning work? - A perspective from Group Theory (2014, background): “This step – executed layer-wise, starting with the ﬁrst hidden layer and sequentially moving deeper – is often referred to as pre-training (see Hinton et al. (2006); Hinton (2007); Salakhutdinov & Hinton (2009); Bengio et al. (in preparation)) and the resulting layer is called an autoencoder .”
Exploring Strategies for Training Deep Neural Networks (2009, methodology): “Hinton (2006) argues that this representation can be improved by giving it as input to another RBM, whose posterior over its hidden layer will then provide a more complex representation of the input.”
Deep unsupervised learning through spatial contrasting (2016, methodology): “The network was later ﬁne-tuned in a supervised manner as displayed by Hinton (2007).”
Learning Deep Architectures for AI (2007, background): “The second level combines these values, typically with a simple linear combination (an OR-like operation), in order to estimate the desired output.”
Generative versus discriminative training of RBMs for classification of fMRI images (2008, methodology): “The methods clearly generalize to arbitrary numbers of classes.”