On the Transfer of Disentangled Representations in Realistic Settings

By Andrea Dittadi, Frederik Trauble, Francesco Locatello, M. Wuthrich, Vaibhav Agrawal, Ole Winther, Stefan Bauer, Bernhard Schölkopf. 2020

89 citations recorded by the source corpus.

Learning meaningful representations that disentangle the underlying structure of the data generating process is considered to be of key importance in machine learning. While disentangled representations were found to be useful for diverse tasks such as abstract reasoning and fair classification, their scalability and real-world impact remain questionable. We introduce a new high-resolution dataset with 1M simulated images and over 1,800 annotated real-world images of the same robotic setup. In contrast to previous work, this new dataset exhibits correlations, a complex underlying structure, and allows to evaluate transfer to unseen simulated and real-world settings where the encoder i) remains in distribution or ii) is out of distribution. We propose new architectures in order to scale disentangled representation learning to realistic high-resolution settings and conduct a large-scale empirical study of disentangled representations on this dataset. We observe that disentanglement is a good predictor for out-of-distribution (OOD) task performance.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Toward Causal Representation Learning (2021, background): “Machine learning models that incorporate or learn structural knowledge of an environment have been shown to be more efﬁcient and generalize better [9], [11], [15], [16], [27], [58], [77], [84], [85], [141], [157], [177], [181], [197], [211], [212], [244], [258], [272], [274].”
Contrastive Learning Inverts the Data Generating Process (2021, methodology): “Existing benchmarks (Klindt et al., 2021; Burgess & Kim, 2018; Gondal et al., 2019; Dittadi et al., 2021) for disentanglement in 3D scenes differ in important aspects to 3DIdent.”
Towards Out-Of-Distribution Generalization: A Survey (2021, background): “…learned causal UnsupervisedRepresentationLearning ( Section 3 ) Disentangled Representation Learning ( Section 3.1 ) [27], [28], [29], [30], [31], [32], [33](Section 3.1) Causal Representation Learning ( Section 3.2 ) [34], [35], [36](Section 3.2) SupervisedLearningModels ( Section 4 ) ( Section…”