Deep Feature Consistent Variational Autoencoder

By Xianxu Hou, LinLin Shen, Ke Sun, Guoping Qiu. 2016

354 citations recorded by the source corpus.

We present a novel method for constructing Variational Autoencoder (VAE). Instead of using pixel-by-pixel loss, we enforce deep feature consistency between the input and the output of a VAE, which ensures the VAE's output to preserve the spatial correlation characteristics of the input, thus leading the output to have a more natural visual appearance and better perceptual quality. Based on recent deep learning works such as style transfer, we employ a pre-trained deep convolutional neural network (CNN) and use its hidden features to define a feature perceptual loss for VAE training. Evaluated on the CelebA face dataset, we show that our model produces better results than other methods in the literature. We also show that our method can produce latent vectors that can capture the semantic information of face expressions and can be used to achieve state-of-the-art performance in facial attribute prediction.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Variational Autoencoders Pursue PCA Directions (by Accident) (2018, background): “As for representation learning, there is a variety of applications, ranging over image interpolation [19], one-shot generalization [35], language models [43], speech transformation [3], and more.”
Variational Inference of Disentangled Latent Concepts from Unlabeled Observations (2017, background): “…models for real datasets (e.g., text, images, etc.) already work with a relatively simple and disentangled prior p ( z ) having no interaction among the latent dimensions (e.g., the standard Gaussian N (0 , I )) (Bowman et al., 2015; Miao et al., 2016; Hou et al., 2017; Zhao et al., 2017).”
Every Smile is Unique: Landmark-Guided Diverse Smile Generation (2018, background): “For instance, in [16] a variational autoencoder adopting a perceptual loss is shown to be effective for synthesizing faces while encoding information about facial expressions.”
Recent Advances and Trends in Multimodal Deep Learning: A Review (2021, methodology): “VAE is also considered a high-resolution network because it is used to design complex models with large datasets.”
On GANs and GMMs (2018, methodology): “Recent advances in deep learning, have also enabled unsupervised learning of full sized images using various models: Variational Auto Encoders [21, 17], PixelCNN [40, 39, 23, 38], Normalizing Flow [9, 8] and Flow GAN [14].”
Generative Latent Flow: A Framework for Non-adversarial Image Generation (2019, background): “It it shown that training with perceptual loss will lead to sharper outputs [31, 9].”
Perceptual Generative Autoencoders (2019, critical): “While previous work has explored the use of perceptual loss for a similar purpose (Hou et al., 2017), it relies on a VGG net pre-trained on ImageNet and provides no theoretical guarantees.”
Persuasive Faces: Generating Faces in Advertisements (2018, methodology): “However, because L2 reconstruction losses have been shown to produce blurry predictions [27], we instead use a perceptual loss similar to [10].”
Deep generative models for reject inference in credit scoring (2019, methodology): “For the Lending Club data set, we use all variables in Table A1 to train all models, while for the Santander data we use a forward selection approach to select the explanatory variables that are included in the reclassiﬁcation, fuzzy parceling and augmentation methods 13 .”
DeepVID: Deep Visual Interpretation and Diagnosis for Image Classifiers via Knowledge Distillation (2019, background): “DFC-VAE (Deep Feature Consistent VAE [32]) is a variant of the regular VAE.”