Weakly-Supervised Disentanglement Without Compromises

By Francesco Locatello, Ben Poole, Gunnar Rätsch, Bernhard Schölkopf, Olivier Bachem, Michael Tschannen. 2020

365 citations recorded by the source corpus.

Intelligent agents should be able to learn useful representations by observing changes in their environment. We model such observations as pairs of non-i.i.d. images sharing at least one of the underlying factors of variation. First, we theoretically show that only knowing how many factors have changed, but not which ones, is sufficient to learn disentangled representations. Second, we provide practical algorithms that learn disentangled representations from pairs of images without requiring annotation of groups, individual factors, or the number of factors that have changed. Third, we perform a large-scale empirical study and show that such pairs of observations are sufficient to reliably learn disentangled representations on several benchmark data sets. Finally, we evaluate our learned representations and find that they are simultaneously useful on a diverse suite of tasks, including generalization under covariate shifts, fairness, and abstract reasoning. Overall, our results demonstrate that weak supervision enables learning of useful disentangled representations in realistic scenarios.

Subfields: Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

In Signal Processing

Citations in the corpus

A Sober Look at the Unsupervised Learning of Disentangled Representations and their Evaluation (2020, methodology): “Finding sample efficient disentanglement scores is an important research direction for practical semi-supervised disentanglement (Locatello et al., 2020b).”
Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICA (2021, methodology): “An estimation procedure which relies on variational autoencoders (VAEs) (Kingma and Welling, 2014) and learned causal mechanisms regularized for sparsity via binary masks.”
Toward Causal Representation Learning (2021, methodology): “It has recently been used for learning causal models [131], modular architectures [29], [85], and disentangled representations [159].”
Towards Causal Representation Learning (2021, background): “Much of the existing work on disentanglement [109, 158, 159, 256, 157, 135, 202, 61] focuses on independent factors”
On the Transfer of Disentangled Representations in Realistic Settings (2020, methodology): “Here we ﬁx k = 1 as it was shown to lead to higher disentanglement by Locatello et al. (2020).”
A Unifying Review of Deep and Shallow Anomaly Detection (2020, background): “Overall, weak forms of supervision or domain priors may be essential for achieving effective solutions in semantic AD tasks that involve high-dimensional data, as has also been found in other unsupervised learning tasks, such as disentanglement [210], [542], [543].”
Is Independence all you need? On the Generalization of Representations Learned from Correlated Data (2020, methodology): “Locatello et al. (2020) showed access to observations which display diﬀerences in a known number of factors of variation (without knowing which ones speciﬁcally) is suﬃcient to learn disentangled representation.”
Contrastive Learning Inverts the Data Generating Process (2021, background): “Recent work has generalized this contribution to VAEs (Khemakhem et al., 2020a; Locatello et al., 2020; Klindt et al., 2021), as well as (invertible-by-construction) energy-based models (Khemakhem et al.”
Deep Learning for Text Style Transfer: A Survey (2020, background): “Theoretically, although disentanglement is impossible without inductive biases or other forms of supervision (Locatello et al. 2019), disentanglement is achievable with some weak signals, such as only knowing how many factors have changed, but not which ones (Locatello et al. 2020).”
Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding (2020, methodology): “Unlike the LOC data [10], for all of these datasets the object categories never change across transitions – reﬂecting natural object permanence.”
Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style (2021, methodology): “This development has sparked a renewed interest in identifiability in the context of deep representation learning [63, 64, 69, 83, 102, 108, 109, 129].”