Weakly Supervised Disentanglement with Guarantees

By Rui Shu, Yining Chen, Abhishek Kumar, Stefano Ermon, Ben Poole. 2019

149 citations recorded by the source corpus.

Learning disentangled representations that correspond to factors of variation in real-world data is critical to interpretable and human-controllable machine learning. Recently, concerns about the viability of learning disentangled representations in a purely unsupervised manner has spurred a shift toward the incorporation of weak supervision. However, there is currently no formalism that identifies when and how weak supervision will guarantee disentanglement. To address this issue, we provide a theoretical framework to assist in analyzing the disentanglement guarantees (or lack thereof) conferred by weak supervision when coupled with learning algorithms based on distribution matching. We empirically verify the guarantees and limitations of several weak supervision methods (restricted labeling, match-pairing, and rank-pairing), demonstrating the predictive power and usefulness of our theoretical framework.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

Citations in the corpus

A Sober Look at the Unsupervised Learning of Disentangled Representations and their Evaluation (2020, background): “(a) Example of encoder consistency (Shu et al., 2020) for one factor of variation: intervening ( ) on a ground-truth factor (or subset of factors) by fixing its value corresponds to fixing a dimension (or subset of dimensions) in the representation.”
Toward Causal Representation Learning (2021, methodology): “As discussed by Schölkopf et al. [219] and Shu et al. [230], different supervision signals may be used to identify subsets of factors.”
Towards Causal Representation Learning (2021, background): “[230], different supervision signals may be used to identify subsets of factors.”
On the Transfer of Disentangled Representations in Realistic Settings (2020, background): “This limitation can be overcome without the need for explicitly labeled data by introducing weak labels (Locatello et al., 2020; Shu et al., 2019).”
Weakly-Supervised Disentanglement Without Compromises (2020, methodology): “The case k = 1 where we actually know which factor of variation is not shared was previously considered in (Bouchacourt et al., 2018; Hosoya, 2019; Shu et al., 2020).”
Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding (2020, methodology): “[11] Rui Shu, Yining Chen, Abhishek Kumar, Stefano Ermon, and Ben Poole.”
Is Independence all you need? On the Generalization of Representations Learned from Correlated Data (2020, methodology): “To address this theoretical limitation, methods have been proposed that do not require explicitly labelled data but only some weak labeling information (Locatello et al., 2020; Shu et al., 2019).”
Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style (2021, background): “This development has sparked a renewed interest in identifiability in the context of deep representation learning [63, 64, 69, 83, 102, 108, 109, 129].”
A Unifying Review of Deep and Shallow Anomaly Detection (2020, background): “Overall, weak forms of supervision or domain priors may be essential for achieving effective solutions in semantic AD tasks that involve high-dimensional data, as has also been found in other unsupervised learning tasks, such as disentanglement [210], [542], [543].”