Stacked What-Where Auto-encoders

By Junbo Jake Zhao, Michaël Mathieu, Ross Goroshin, Yann LeCun. 2015

263 citations recorded by the source corpus.

We present a novel architecture, the "stacked what-where auto-encoders" (SWWAE), which integrates discriminative and generative pathways and provides a unified approach to supervised, semi-supervised and unsupervised learning without relying on sampling during training. An instantiation of SWWAE uses a convolutional net (Convnet) (LeCun et al. (1998)) to encode the input, and employs a deconvolutional net (Deconvnet) (Zeiler et al. (2010)) to produce the reconstruction. The objective function includes reconstruction terms that induce the hidden states in the Deconvnet to be similar to those of the Convnet. Each pooling layer produces two sets of variables: the "what" which are fed to the next layer, and its complementary variable "where" that are fed to the corresponding layer in the generative decoder.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Generative Models

In Signal Processing

Citations in the corpus

Self-Supervised Learning of Visual Features through Embedding Images into Text Topic Spaces (2017, neutral): “A number of unsupervised algorithms have been proposed to that effect, such as sparse-coding, restricted Boltzmann machines (RBMs), auto-encoders [44], and K-means clustering [4, 8, 20].”
Improved Techniques for Training GANs (2016, neutral): “Stacked What-Where Auto-Encoder [27] 23.”
Unsupervised Embedding Learning via Invariant and Spreading Instance Feature (2019, background): “The performances of some state-of-the-art unsupervised methods (k-MeansNet [5], HMP [2], Satck [54] and Exemplar [8]) are also reported.”
Scattering Networks for Hybrid Representation Learning (2018, methodology): “We use at training and evaluation the predeﬁned 10 folds of 1000 training images each, as given in [61].”
Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks (2015, methodology): “…popular method is to train auto-encoders (convolutionally, stacked (Vincent et al., 2010), separating the what and where components of the code (Zhao et al., 2015), ladder structures (Rasmus et al., 2015)) that encode an image into a compact code, and decode the code to reconstruct the image…”
Adversarially Learned Inference (2016, neutral): “02% SWWAE without dropout (Zhao et al. , 2015) 27 .”
Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning (2017, methodology): “SWWAE [43] 23.56 Skip Generative Model [24] 16.30 Ladder networks, Γ model [30] 20.40 CatGAN [33] 19.58 GAN with feature matching [32] 8.11 18.63 model [21] 5 Table 5 shows the performance of VAT and contemporary methods implemented with moderate image data augmentation (translation and horizontal…”
Energy-based Generative Adversarial Network (2016, background): “We leave the usage of upsampling or switches-unpooling (Zhao et al., 2015) to future research.”
Semi-supervised Learning with Ladder Networks (2015, background): “Zhao et al. (2015) use a similar structure and call it the stacked what-where autoencoder (SWWAE).”
Autoencoding beyond pixels using a learned similarity metric (2015, background): “Unfortunately, we have not been able to reach results competitive with the state-of-the-art (Rasmus et al., 2015; Zhao et al., 2015).”
Unsupervised Learning by Predicting Noise (2017, background): “The decoder is often a fully connected network (Ranzato et al., 2007) or a deconvolutional network (Masci et al., 2011; Zhao et al., 2016) but can be more sophisticated, like a PixelCNN network (van den Oord et al., 2016).”
Neural Photo Editing with Introspective Adversarial Networks (2016, neutral): “02% SWWAE with dropout (Zhao et al., 2015) 23 .”
Stacked Hourglass Networks for Human Pose Estimation (2016, background): “[43] develop a unified framework for supervised, unsupervised and semi-supervised learning by adding a reconstruction loss.”
Stacked Generative Adversarial Networks (2016, neutral): “Also, some auto-encoder architectures employ layer-wise reconstruction loss [60, 48, 67, 66].”
Deep spatial autoencoders for visuomotor learning (2015, methodology): “Some spatial network architectures have been proposed for tasks such as video prediction and dynamics modeling [30], [21].”
Learning to Navigate in Complex Environments (2016, background): “Recently, the incorporation of additional objectives, designed to augment representation learning through auxiliary reconstructive decoding pathways (Zhang et al., 2016; Rasmus et al., 2015; Zhao et al., 2015; Mirowski et al., 2010), has yielded beneﬁts in large scale classiﬁcation tasks.”