A Spectral Regularizer for Unsupervised Disentanglement

By Aditya Ramesh, Youngduck Choi, Yann LeCun. 2018

44 citations recorded by the source corpus.

A generative model with a disentangled representation allows for independent control over different aspects of the output. Learning disentangled representations has been a recent topic of great interest, but it remains poorly understood. We show that even for GANs that do not possess disentangled representations, one can find curved trajectories in latent space over which local disentanglement occurs. These trajectories are found by iteratively following the leading right-singular vectors of the Jacobian of the generator with respect to its input. Based on this insight, we describe an efficient regularizer that aligns these vectors with the coordinate axes, and show that it can be used to induce disentangled representations in GANs, in a completely unsupervised manner.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

In Representation & Self-Supervised Learning

Citations in the corpus

GAN Inversion: A Survey (2021, methodology): “Optimization-based GAN inversion typically optimizes the latent code based on either gradient descent [24], [25], [44], [44], [109], [109], [110] or other iterative algorithms [95], [111].”
Low-Rank Subspaces in GANs (2021, unknown): “Although there are also some works using the Jacobian matrix to analyze the GAN latent space [26, 7, 34], the low-rank subspace enables precise generation control by providing the indispensable null space.”
Orthogonal Jacobian Regularization for Unsupervised Disentanglement in Image Generation (2021, methodology): “Instead of post-processing, studies have also been given to achieve disentanglement by incorporating extra regularization [7, 10, 30, 32, 34, 40, 48] in GAN training.”
Do Not Escape From the Manifold: Discovering the Local Coordinates on the Latent Space of GANs (2021, methodology): “The local methods (e.g. Ramesh et al. (2018), Latent Mapper in StyleCLIP (Patash-nik et al., 2021), and attribute-conditioned normalizing ﬂow in StyleFlow (Abdal et al., 2021)) suggest a sample-wise traversal direction.”
Unsupervised Discovery of Interpretable Directions in the GAN Latent Space (2020, methodology): “An alternative line of research on the model interpretability aims to train generators with disentangled latent spaces (Chen et al., 2016; Higgins et al., 2017; Liu et al., 2019; Lee et al., 2020; Ramesh et al., 2018).”