Unsupervised learning of hierarchical representations with convolutional deep belief networks

By Honglak Lee, Roger Baker Grosse, Rajesh Ranganath, Andrew Y. Ng. 2011

397 citations recorded by the source corpus.

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

Citations in the corpus

Deep Learning: Methods and Applications (2014, background): “A further example of hybrid deep networks is the use of generative models of DBNs to pre-train deep convolutional neural networks (deep CNNs) [215, 216, 217].”
Learning Invariant Representations with Local Transformations (2012, methodology): “Specifically, convolutional training (LeCun et al., 1989; Kavukcuoglu et al., 2010; Zeiler et al., 2010; Lee et al., 2011; Sohn et al., 2011) is one of the most popular methods that encourages shift-invariance during the feature learning.”
VConv-DAE: Deep Volumetric Shape Learning Without Object Labels (2016, methodology): “Despite the impressive and promising results of such Deep belief nets [6,7], these models can be challenging to train.”
DeepTox: Toxicity Prediction using Deep Learning (2016, neutral): “Images adopted from Lee et al. (2011) with permission from the authors.”
Computational Phenotype Discovery Using Unsupervised Feature Learning over Noisy, Sparse, and Irregular Clinical Data (2013, background): “For example, deep learning produces edge and spot detectors as translation-invariant features for photographic images [23], stereotyped object parts as features for images of objects [23], and phonemes as features from speech recordings [24].”
Improving object detection with deep convolutional networks via Bayesian optimization and structured prediction (2015, background): “Motivated by the recent success of deep learning [26, 21, 3, 6, 27, 4, 35] on visual object recognition tasks [25, 38, 46, 39, 42], significant improvements have been made in the object detection problem [41, 11, 20].”
SampleFix: Learning to Correct Programs by Sampling Diverse Fixes (2019, background): “Advances in machine learning, deep learning (Lecun et al., 1998; Lee et al., 2011; Krizhevsky et al., 2012), computer vision (Girshick, 2015; Simonyan and Zisserman, 2015) and NLP (Sutskever et al.”
3 D ShapeNets : A Deep Representation for Volumetric Shape Modeling (2015, methodology): “[20]), we do not use any form of pooling in the hidden layers – while pooling may enhance the invariance properties for recognition, in our case, it would also lead to greater uncertainty during reconstruction.”
SenticNet 4: A Semantic Resource for Sentiment Analysis Based on Conceptual Primitives (2016, methodology): “In order to emulate such a process, we use simple but powerful meta-algorithms which underlie neuronal learning (Lee et al., 2011).”
Learning Structured Output Representation using Deep Conditional Generative Models (2015, background): “Along with the recent breakthroughs in supervised deep learning methods, there has been a progress in deep generative models, such as deep belief networks [10, 20] and deep Boltzmann machines [25].”
Learning and Selecting Features Jointly with Point-wise Gated Boltzmann Machines (2013, background): “This construction makes sense because the first layer features are mostly generic, and the class-specific features emerge in higher layers (Lee et al., 2011).”
Deep Learning in Spiking Neural Networks (2018, neutral): “DBNs and convolutional DBNs have successfully been employed inmany areas such as visual processing (Krizhevsky & Hinton, 2010; Liu, Han, Meng, & Tong, 2014; Mleczko, Kapuściński, & Nowicki, 2015; Susskind, Hinton, Movellan, &Anderson, 2008), audio processing (Hamel& Eck, 2010; Kang, Qian, & Meng, 20”
Deep learning for robust feature generation in audiovisual emotion recognition (2013, background): “Unsupervised Feature Learning and Deep Learning Deep learning techniques (See [9] for a survey) have become increasingly popular in various communities including speech and language processing [10] [11] [12] and vision processing [26] [27] [28] [29] [30].”
Scalable High Performance Image Registration Framework by Unsupervised Deep Feature Representations Learning (2015, methodology): “Speciﬁcally, the Kullback–Leibler divergence [37], [40], [42] is used to impose the sparsity constraint to each hidden node by enforcing the average activation degree over the whole training data, i.e., ˆ ρ j = 1 M (cid:2) =1 h m ( j ) , to be close to a very small value ρ (here, ρ is set to 0.001…”
Deep Learning for Computer Vision: A Brief Review (2018, background): “However, a later variation of the DBN, the Convolutional Deep Belief Network (CDBN) ([42, 43]), uses the spatial information of neighboring pixels by introducing convolutional RBMs, thus producing a translation invariant generative model that successfully scales when it comes to high dimensional ima”
AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks (2018, background): “This is promising and reasonable because learning hierarchical representation has proven quite effective in computer vision and speech processing with deep neural networks [3, 18].”
3D ShapeNets: A deep representation for volumetric shapes (2014, methodology): “[21]), we do not use any form of pooling in the hidden layers – while pooling may enhance the invariance properties for recognition, in our case, it would also lead to greater uncertainty for shape reconstruction.”