Unsupervised Model Selection for Variational Disentangled Representation Learning

By Sunny Duan, Nicholas Watters, Loic Matthey, Christopher P. Burgess, Alexander Lerchner, Irina Higgins. 2019

89 citations recorded by the source corpus.

Disentangled representations have recently been shown to improve fairness, data efficiency and generalisation in simple supervised and reinforcement learning tasks. To extend the benefits of disentangled representations to more complex domains and practical applications, it is important to enable hyperparameter tuning and model selection of existing unsupervised approaches without requiring access to ground truth attribute labels, which are not available for most datasets. This paper addresses this problem by introducing a simple yet robust and reliable method for unsupervised disentangled model selection. Our approach, Unsupervised Disentanglement Ranking (UDR), leverages the recent theoretical results that explain why variational autoencoders disentangle (Rolinek et al, 2019), to quantify the quality of disentanglement by performing pairwise comparisons between trained model representations. We show that our approach performs comparably to the existing supervised alternatives across 5,400 models from six state of the art unsupervised disentangled representation learning model classes. Furthermore, we show that the ranking produced by our approach correlates well with the final task performance on two different domains.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICA (2021, methodology): “4, we select αz and αa using an adaptation of the unsupervised model selection criterion proposed by Duan et al. (2020).”
Weakly-Supervised Disentanglement Without Compromises (2020, background): “…of disentangled representations (Higgins et al., 2017a; Burgess et al., 2018; Kim & Mnih, 2018; Chen et al., 2018; Kumar et al., 2018) along with quantitative evaluation (Kim & Mnih, 2018; Eastwood & Williams, 2018; Kumar et al., 2018; Chen et al., 2018; Ridgeway & Mozer, 2018; Duan et al., 2019).”
Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding (2020, methodology): “To ﬁnd the best hyperparameters for the conditional prior regularization and the prior rate, we perform a random search over γ ∈ [1 , 16] and λ ∈ [1 , 10] and compute the recently proposed unsupervised disentanglement ranking (UDR) scores for unsupervised model selection [55].”
Efficient-VDVAE: Less is more (2022, background): “Fortunately, in practice, VAEs (and HVAEs) train in a polarized regime and learn a latent space where the majority of the latent dimensions are "turned off" for all samples [51, 52].”