Challenges in Disentangling Independent Factors of Variation

By Attila Szabó, Qiyang Hu, Tiziano Portenier, Matthias Zwicker, Paolo Favaro. 2017

52 citations recorded by the source corpus.

We study the problem of building models that disentangle independent factors of variation. Such models could be used to encode features that can efficiently be used for classification and to transfer attributes between different images in image synthesis. As data we use a weakly labeled training set. Our weak labels indicate what single factor has changed between two data samples, although the relative value of the change is unknown. This labeling is of particular interest as it may be readily available without annotation costs. To make use of weak labels we introduce an autoencoder model and train it through constraints on image pairs and triplets. We formally prove that without additional knowledge there is no guarantee that two images with the same factor of variation will be mapped to the same feature. We call this issue the reference ambiguity. Moreover, we show the role of the feature dimensionality and adversarial training. We demonstrate experimentally that the proposed model can successfully transfer attributes on several datasets, but show also cases when the reference ambiguity occurs.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesis (2021, background): “In a conditional setting [36], latent variables which are independent from the conditioning lead to disentangled representations [28, 66, 45, 57].”
Disentangling Latent Hands for Image Synthesis and Pose Estimation (2018, background): “Several of these works [24, 28, 29, 34], however, require specially designed layers and loss functions, making the architectures difficult to work with and extend beyond their intended task.”
Latent Optimization for Non-adversarial Representation Disentanglement (2019, methodology): “[26]: This method trains an autoencoder similar to [4], however the adversarial loss is not set on the codes but rather on pairs of images.”
Guided Variational Autoencoder for Disentanglement Learning (2020, methodology): “Ad-versarial training has been adopted in approaches such as [41, 48].”
Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding (2020, background): “[43] Attila Szabó, Qiyang Hu, Tiziano Portenier, Matthias Zwicker, and Paolo Favaro.”
Disentangling Factors of Variation with Cycle-Consistent Variational Auto-Encoders (2018, methodology): “[8] have also explored a non-adversarial approach to disentangle factors of variation.”
Disentangled representation learning in cardiac image analysis (2019, background): “[2016] and Szabó et al. [2018], which both learn a factorisation into known and other residual factors.”