Disentangling by Factorising

By Hyunjik Kim, Andriy Mnih. 2018

1,568 citations recorded by the source corpus.

We define and address the problem of unsupervised learning of disentangled representations on data generated from independent factors of variation. We propose FactorVAE, a method that disentangles by encouraging the distribution of representations to be factorial and hence independent across the dimensions. We show that it improves upon $\beta$-VAE by providing a better trade-off between disentanglement and reconstruction quality. Moreover, we highlight the problems of a commonly used disentanglement metric and introduce a new metric that does not suffer from them.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

A Sober Look at the Unsupervised Learning of Disentangled Representations and their Evaluation (2020, methodology): “The FactorVAE (Kim and Mnih, 2018) and the β-TCVAE (Chen et al., 2018) penalize the total correlation (Watanabe, 1960) with adversarial training (Nguyen et al., 2010; Sugiyama et al., 2012) or with a tractable but biased Monte-Carlo estimator respectively.”
A Style-Based Generator Architecture for Generative Adversarial Networks (2018, background): “Unfortunately the metrics recently proposed for quantifying disentanglement [25, 30, 6, 18] require an encoder network that maps input images to latent codes.”
Dual Contradistinctive Generative Autoencoder (2020, neutral): “33 FactorVAE(γ=5) [36] 6.”
Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges (2021, methodology): “If we use an unsupervised algorithm for disentangling the space, and then evaluate whether it indeed disentangled the known concepts, then we have a useful quantitative evaluation (Higgins et al., 2017; Chen et al., 2018b; Kim and Mnih, 2018; Eastwood and Williams, 2018; Do and Tran, 2019).”
Reconciling deep learning with symbolic artificial intelligence: representing objects and relations (2019, background): “One way to address this issue is to manipulate the loss function being minimised by the learning algorithm so that it favours disentangled representations [20,21 ,22,23] (Figure 1b bottom).”
Flexibly Fair Representation Learning by Disentanglement (2019, methodology): “Chen et al. (2018); Kim & Mnih (2018) and Esmaeili et al. (2019) argue that decomposing the ELBO and penalizing the total correlation increases disentanglement in the latent representations.”
Towards Causal Representation Learning (2021, background): “Much of the existing work on disentanglement [109, 158, 159, 256, 157, 135, 202, 61] focuses on independent factors”
On the Transfer of Disentangled Representations in Realistic Settings (2020, background): “…to p ( z ), diﬀerent regularization strategies have been proposed, along with evaluation metrics to gauge the disentanglement of the learned representations (Higgins et al., 2017a; Kim and Mnih, 2018; Burgess et al., 2018; Kumar et al., 2018; Chen et al., 2018; Eastwood and Williams, 2018).”
Towards causal generative scene models via competition of experts (2020, background): “…(Bengio et al., 2013) has largely focused on disentangling factors of variation within a single shared representation space, e.g., by training a large encoder-decoder architecture with different forms of regularization (Higgins et al.; Kim & Mnih, 2018; Chen et al., 2018; Locatello et al., 2019).”
Weakly-Supervised Disentanglement Without Compromises (2020, methodology): “We replicate the experiment of van Steenkiste et al. (2019) on Shapes3D under the same exact experimental conditions (see Appendix B for more details).”
Learning Representations by Maximizing Mutual Information in Variational Autoencoders (2019, background): “However, such an estimation is computationally expensive and also limits the performance benefits [19].”
Weakly Supervised Disentanglement with Guarantees (2019, methodology): “This is in contrast to the existing disentanglement deﬁnitions and metrics, which require access to the ground truth factors (Higgins et al., 2017; Kumar et al., 2018; Kim & Mnih, 2018; Chen et al., 2018a; Suter et al., 2018; Ridgeway & Mozer, 2018; Eastwood & Williams, 2018).”
On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Dataset (2019, background): “, FactorVAE [19] or β-TCVAE [5], while DIP-VAE proposed to encourage factorization of the inferred prior qφ(z) = ∫ qφ(z|x)p(x) dx.”
On the Fairness of Disentangled Representations (2019, methodology): “On Shapes3D we observe a similar result, but the correlation between the two groups of scores is stronger than on SmallNORB.”
Are Disentangled Representations Helpful for Abstract Visual Reasoning? (2019, methodology): “[52] studied several of these metrics, which we will adopt for our purposes in this work: the BetaVAE score [27], the FactorVAE score [42], the Mutual Information Gap (MIG) [10], the disentanglement score from Eastwood et al.”
Multi-Object Representation Learning with Iterative Variational Inference (2019, methodology): “This includes disentangled representations through the use of β -VAEs (Higgins et al., 2017a), adversarial autoencoders (Makhzani et al., 2015), Factor VAEs (Kim & Mnih, 2018), and improved generalization through non-euclidean embed-dings (Nickel & Kiela, 2017).”
MONet: Unsupervised Scene Decomposition and Representation (2019, background): “Recent progress has been made on learning object representations that support feature compositionality, for which a variety of VAE-based methods have become state-of-the-art [Higgins et al., 2017, Kim and Mnih, 2017, Chen et al., 2016, Locatello et al., 2018].”
Variational Autoencoders Pursue PCA Directions (by Accident) (2018, methodology): “With this in mind, it makes perfect sense that recent improvements of (β-)VAE [6, 21, 40] incorporate additional terms promoting precisely independence.”
Generalization in anti-causal learning (2018, background): “Based on the previous paragraph, it makes sense that most state-of-the-art methods for disentangling are based on autoencoder architectures (Higgins et al., 2017; Kim and Mnih, 2018; Kumar et al., 2017).”