Disentangling Factors of Variation with Cycle-Consistent Variational Auto-Encoders

By Ananya Harsh Jha, Saket Anand, Maneesh Kumar Singh, V. S. Rao Veeravasarapu. 2018

137 citations recorded by the source corpus.

Generative models that learn disentangled representations for different factors of variation in an image can be very useful for targeted data augmentation. By sampling from the disentangled latent subspace of interest, we can efficiently generate new data necessary for a particular task. Learning disentangled representations is a challenging problem, especially when certain factors of variation are difficult to label. In this paper, we introduce a novel architecture that disentangles the latent space into two complementary subspaces by using only weak supervision in form of pairwise similarity labels. Inspired by the recent success of cycle-consistent adversarial architectures, we use cycle-consistency in a variational auto-encoder framework. Our non-adversarial approach is in contrast with the recent works that combine adversarial training with auto-encoders to disentangle representations. We show compelling results of disentangled latent subspaces on three datasets and compare with recent works that leverage adversarial training.

Subfields: Computer Vision and Pattern Recognition, Media Technology, Generative Models, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

In Generative Models

In Representation & Self-Supervised Learning

Citations in the corpus

Theory and Evaluation Metrics for Learning Disentangled Representations (2019, methodology): “Other methods that are derived from [29] include [14, 38].”
Demystifying Inter-Class Disentanglement (2019, methodology): “For CelebA, given Classiﬁcation experiments: To assess the disentanglement of our learned representations, we follow the protocol in Harsh Jha et al. (2018) and train a classiﬁer to classify class labels from content codes and vice versa.”
Behavior-Driven Synthesis of Human Dynamics (2021, background): “Existing works for static images typically resort to supervision by exploiting pairs of data samples sharing one factor while differing in the remaining factors [42, 26], which allows for a natural disentanglement signal.”
Guided Variational Autoencoder for Disentanglement Learning (2020, background): “Disentanglement learning [41, 48, 23, 1, 16, 26] recently becomes a popular topic in representation learning.”
The Human Tumor Atlas Network: Charting Tumor Transitions across Space and Time at Single-Cell Resolution (2020, unknown): “Progress on similar problems in other computational domains could provide guidance (Jha et al., 2018; Ma et al., 2019; Wang et al., 2016).”
Swapping Autoencoder for Deep Image Manipulation (2020, methodology): “First, while most require human supervision, such as class labels [55], pairwise image similarity [35], or object locations [68], our method is fully unsupervised.”
Disentangling Latent Hands for Image Synthesis and Pose Estimation (2018, background): “To force the cross-modality pairings onto a single latent space, separate VAEs are learned in an alternating fashion, with one input modality contributing to the loss per iteration.”
Latent Optimization for Non-adversarial Representation Disentanglement (2019, methodology): “On CelebA, [8], was unable to transfer the pose.”
Orthogonal Jacobian Regularization for Unsupervised Disentanglement in Image Generation (2021, background): “Variational Autoencoder (VAE) [26] has been widely adopted in state-of-the-art disentanglement methods [6, 9, 15, 18, 21, 25, 27, 28].”