Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction

By Jonathan Masci, Ueli Meier, Dan Claudiu Ciresan, J. Urgen Schmidhuber. 2011

2,139 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Unsupervised Representation Learning by Predicting Image Rotations (2018, methodology): “…methods (Dosovitskiy et al., 2014; Liao et al., 2016; Yang et al., 2016), reconstruction based methods (Bengio et al., 2007; Huang et al., 2007; Masci et al., 2011), and methods that involve learning generative probabilistic models Goodfellow et al. (2014); Donahue et al. (2016); Radford et…”
Conditional Image Generation with PixelCNN Decoders (2016, methodology): “Starting with a traditional convolutional auto-encoder architecture [16], we replace the deconvolutional decoder with a conditional PixelCNN and train the complete network end-to-end.”
Scaling and Benchmarking Self-Supervised Visual Representation Learning (2019, background): “This can be modeled as maximizing the probability of reconstructing the input [43, 51, 67] and optionally estimating latent variables [29, 58] or using adversarial training [15, 44].”
Generative Moment Matching Networks (2015, background): “For example, it may be possible to use a GMMN+AE with convolutional auto-encoders (Zeiler et al., 2010; Masci et al., 2011; Makhzani & Frey, 2014) in order to create generative models of high resolution color images.”
Deep Clustering for Unsupervised Learning of Visual Features (2018, background): “Typically, a parametrized mapping is learned between a predefined random noise and the images, with either an autoencoder [4, 22, 29, 40, 62], a generative adversarial network (GAN) [20] or more directly with a reconstruction loss [6].”
Unsupervised Learning by Predicting Noise (2017, methodology): “Many have also used autoencoders with a reconstruction loss (Bengio et al., 2007; Ranzato et al., 2007; Masci et al., 2011).”
Stacked What-Where Auto-encoders (2015, background): “Convolutional auto-encoder variants also relate to our work [10, 18, 19].”
Deep Convolutional Inverse Graphics Network (2015, methodology): “Models such as Convolutional Neural Networks (CNNs) [16, 13], Restricted Boltzmann Machines (RBM) based generative models [9, 25, 17], and Auto-encoders [1, 29, 20] have been successfully applied to produce multiple layers of increasingly abstract visual representations.”
Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks (2015, neutral): “, dimensionality reduction, feature engineering, or network pre-training [21], [27], [28]).”
A practical tutorial on autoencoders for nonlinear feature fusion: Taxonomy, models, software and guidelines (2018, neutral): “Convolutional autoencoder [75].”
Latent Space Physics: Towards Learning the Temporal Evolution of Fluid Flow (2018, methodology): “2013] for encoder and decoder to create a deep convolutional autoencoder [Masci et al. 2011].”
The Challenges of Continuous Self-Supervised Learning (2022, background): “Self-supervised learning has a long history in computer vision [7, 35, 42, 50, 70, 71] aiming to learn representations of visual data by solving tasks that can be defined without human annotations.”
Understanding Visual Concepts with Continuation Learning (2016, methodology): “This model is a deep convolutional autoencoder (Hinton and Salakhutdinov 2006; Bengio 2009; Masci et al. 2011) with modiﬁcations to accommodate multiple frames and encourage a particular factorization in the latent space.”
Augmenting Supervised Neural Networks with Unsupervised Objectives for Large-scale Image Classification (2016, methodology): “With pooling-induced sparsity bottlenecks (Makhzani & Frey, 2015), the convolutional SAE (Masci et al., 2011) can learn features from middle-size images.”
VConv-DAE: Deep Volumetric Shape Learning Without Object Labels (2016, background): “In order to share weights for stationary distributions such as they occur in images, convolutional auto-encoders have been proposed [18].”
Spatio-temporal video autoencoder with differentiable memory (2015, background): “In a classic spatial autoencoder Masci et al. (2011), the encoder and decoder learn proprietary feature spaces that allow an optimal decomposition of the input using some form of regularisation to prevent learning a trivial mapping.”