Learning Hierarchical Features from Deep Generative Models

By Shengjia Zhao, Jiaming Song, Stefano Ermon. 2017

111 citations recorded by the source corpus.

Subfields: Generative Models, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Generative Models

In Representation & Self-Supervised Learning

Citations in the corpus

Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images (2020, background): “A.5 A NOTE ON LEARNING HIERARCHICAL FEATURES The work of Zhao et al. (2017) may appear to contradict our work, by suggesting that additional layers in hierarchical VAEs do not lead to additional expressivity, based off their finding that Gibbs sampling from the last stochastic layer is sufficient…”
Recent Advances in Autoencoder-Based Representation Learning (2018, methodology): “…Information dropout (15) [10], DC-IGN [11], FaderNetworks (18) [12], VFAE (17) [13] Hierarchical representation 1 PixelVAE [14], LVAE [15], VLaAE [16], Semi-supervised VAE [17], PixelGAN-AE [18], VLAE [19], VQ-VAE [20] Semi-supervised learning Semi-supervised VAE [17], [21], PixelGAN-AE (14)…”
Bit-Swap: Recursive Bits-Back Coding for Lossless Compression with Hierarchical Latent Variables (2019, background): “Especially when using deep hierarchies of latent variables, the higher latent layers can become redundant and therefore unused (Zhao et al., 2017).”
Factorizable Graph Convolutional Networks (2020, background): “…in disentangled representation learning are based on the architecture of auto-encoder [Higgins et al., 2017, Feng et al., 2018, Bouchacourt et al., 2018, Burgess et al., 2018, Wang et al., 2017, Kim and Mnih, 2018] or generative model [Chen et al., 2016, Zhao et al., 2017, Siddharth et al., 2017].”
DarkneTZ: towards model privacy at the edge using trusted execution environments (2020, background): “formation that is more specificto the classification task (e.g., face identity). The memorization difference per layer has been verified both in convolutional layers [60, 62] and in generative models [65]. Evidently, an untrusted party with access to the model can leverage the memorized information ”
Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models (2021, background): “It has been argued that a single level of latents is sufficient since Gibbs sampling performed on that level can recover the data distribution [257].”
Degeneration in VAE: in the Light of Fisher Information Loss (2018, background): “Although in some previous work like Zhao et al. (2017) the mutual information has been proposed to enhance the connection between data and latent code, as the encoder and decoder go deeper, mutual information between data and latent code is more difficult to maintain.”
Structural Autoencoders Improve Representations for Generation and Transfer (2020, background): “Our more structured architectures are related to the Variational Ladder autoencoder (VLAE) [16] which separates the latent space into separate chunks each of which is processed at different levels of the encoder and decoder.”