Efficient Learning of Sparse Representations with an Energy-Based Model

By Marc'Aurelio Ranzato, Christopher S. Poultney, Sumit Chopra, Yann LeCun. 2006

1,404 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Generative Models, Representation & Self-Supervised Learning, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Generative Models

In Representation & Self-Supervised Learning

In Signal Processing

Citations in the corpus

Deep Generative Stochastic Networks Trainable by Backprop (2013, background): “, 2008), and sparse coding variants (Lee et al., 2007; Ranzato et al., 2007).”
GSNs : Generative Stochastic Networks (2015, neutral): “, 2008), and sparse coding variants (Lee et al., 2007; Ranzato et al., 2007).”
Pedestrian Detection with Unsupervised Multi-stage Feature Learning (2012, background): “In [16], it was shown that unsupervised learning can be used to train deep hierarchical models and the final representation achieved is actually useful for a variety of different tasks [32, 24, 4].”
Representation Learning: A Review and New Perspectives (2012, neutral): “…earliest uses of single-layer auto-encoders for building deep architectures by stacking them (Bengio et al., 2007) considered the idea of tying the encoder weights and decoder weights to restrict capacity as well as the idea of introducing a form of sparsity regularization (Ranzato et al., 2007).”
Multi-column deep neural networks for image classification (2012, neutral): “40 CNN [28] 0.”
Energy-based Generative Adversarial Network (2016, methodology): “L G ] 6 M ar 2 01 7”
Deep, Big, Simple Neural Nets for Handwritten Digit Recognition (2010, methodology): “This is signiﬁcantly better than the best previously published results, namely, 0.39% by Ranzato et al. (2006) and 0.40% by Simard et al. (2003), both obtained by more complex methods.”
Fast Inference in Sparse Coding Algorithms with Applications to Object Recognition (2010, neutral): “Many sparse coding algorithms have been shown to learn good local feature extra ctors for natural images [4, 5, 6, 7, 8].”
Deep learning in neural networks: An overview (2014, neutral): “39% (Ranzato et al., 2006), using training pattern deformations (Sec.”
Relational autoencoder for feature extraction (2017, background): “Poultney et al. [43] presented an idea to increase the sparsity of network structure so as to limit the increasing of weights.”
Deep Learning: Methods and Applications (2014, background): “[297, 295] proposed an unsupervised learning algorithm called sparse encoding symmetric machine (SESM), which is quite similar to RBM.”
Deep Multimodal Feature Analysis for Action Recognition in RGB+D Videos (2016, methodology): “It has been shown in [29, 50] that applying sparsity on the features of Y and Z will help to improve the learning capability, especially when components are overcomplete.”
Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network (2016, methodology): “[27] suggested a method for SISR based on an extension of the predictive convolutional sparse coding framework [29].”
Practical Recommendations for Gradient-Based Training of Deep Architectures (2012, methodology): “Another option is to penalize the biases of the hidden units, to make them more negative (Ranzato et al. , 2007; Lee et al. , 2008; Goodfellow et al. , 2009; Larochelle and Bengio, 2008).”
Recent Advances in Autoencoder-Based Representation Learning (2018, background): “Early approaches to learn abstract representations using autoencoders include stacking single-layer autoencoders [64] to build deep architectures and imposing a sparsity prior to the latent variables [65].”
Learning Invariant Representations with Local Transformations (2012, background): “In recent years, unsupervised feature learning algorithms have emerged as promising tools for learning representations from data (Hinton et al., 2006; Bengio et al., 2007; Ranzato et al., 2006).”
Deep Learning of Representations: Looking Forward (2013, methodology): “…of research was kick-started in 2006 by a few research groups, starting with Geoff Hinton’s group, who initially focused on stacking unsupervised representation learning algorithms to obtain deeper representations (Hinton et al., 2006; Bengio et al., 2007; Ranzato et al., 2007; Lee et al., 2008).”