Efficient sparse coding algorithms

By Honglak Lee, Alexis Battle, Rajat Raina, Andrew Y. Ng. 2006

2,878 citations recorded by the source corpus.

Subfields: Biophysics, Cognitive Neuroscience, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biophysics

In Cognitive Neuroscience

In Representation & Self-Supervised Learning

Citations in the corpus

Deep Generative Stochastic Networks Trainable by Backprop (2013, methodology): “Gaussian mixture rather than a Bernoulli mixture to compute the likelihood, but we can compare with Rifai et al. (2012); Bengio et al.”
Unsupervised Visual Representation Learning by Context Prediction (2015, methodology): “Sparse autoencoders also use reconstruction as a pretext task, along with a sparsity penalty [39], and such autoencoders may be stacked to form a deep representation [32, 31].”
GSNs : Generative Stochastic Networks (2015, neutral): “, 2008), and sparse coding variants (Lee et al., 2007; Ranzato et al., 2007).”
Pedestrian Detection with Unsupervised Multi-stage Feature Learning (2012, background): “In [16], it was shown that unsupervised learning can be used to train deep hierarchical models and the final representation achieved is actually useful for a variety of different tasks [32, 24, 4].”
Building high-level features using large scale unsupervised learning (2011, methodology): “Our work is inspired by recent successful algorithms in unsupervised feature learning and deep learning [20, 9, 1, 24, 18].”
Fast Inference in Sparse Coding Algorithms with Applications to Object Recognition (2010, methodology): “(2) with [7], and “Regressor ” that is separately trained to approximate the exact optimal codes p roduced by FS.”
Tutorial on Variational Autoencoders (2016, neutral): “sparse autoencoders [10, 11] or denoising autoencoders [12, 13].”
Recent advances in convolutional neural networks (2015, methodology): “In this section, we introduce some fast processing methods of CNNs.”
The Challenges of Continuous Self-Supervised Learning (2022, methodology): “coding [36, 54, 55], inpainting [59] and colorization [18, 34, 86], to methods that learn representations predictive of spatial context [19, 22, 53], temporal context [21, 47, 58, 61, 79, 80], or concurrent modalities like audio [4, 52, 57, 70], text [17, 23, 62] or speech [43, 44].”
Rethinking Pre-training and Self-training (2020, background): “The resurgence of deep learning in the 2000s also began with unsupervised pre-training [22–26].”
Image Super-Resolution Using Deep Convolutional Networks (2014, methodology): “Then the sparse coding solver, like Feature-Sign [28], will first project the patch onto a (low-resolution) dictionary.”
HOGgles: Visualizing Object Detection Features (2013, background): “6 exist [14, 11], we can invert features in under two seconds on a 4 core CPU.”
Bag of visual words and fusion methods for action recognition: Comprehensive study and good practice (2014, neutral): “The l1-norm relaxation allows for more efficient optimization algorithm [27] and obtaining the global optimal solution.”
Cooperative Learning of Audio and Video Models from Self-Supervised Synchronization (2018, background): “Inspirational work in this area includes deep belief networks [33], stacked autoencoders [34], shift-invariant decoders [35], sparse coding [36], TICA [37], stacked ISAs [38].”
Low-Rank Approximations for Conditional Feedforward Computation in Deep Neural Networks (2013, background): “Such a penalty is commonly used in sparse dictionary learning algorithms and tends to push elements of al towards zero [13].”
Optimization with Sparsity-Inducing Penalties (2011, background): “, [91]), and here takes specific advantage of sparsity from a computational point of view [9, 56, 69, 92, 102, 104, 113], since the subproblems that need to be solved are typically much smaller than the original one.”