Independently Controllable Factors

By Valentin Thomas, Jules Pondard, Emmanuel Bengio, Marc Sarfati, Philippe Beaudoin, Marie-Jean Meurs, Joelle Pineau, Doina Precup, Yoshua Bengio. 2017

71 citations recorded by the source corpus.

It has been postulated that a good representation is one that disentangles the underlying explanatory factors of variation. However, it remains an open question what kind of training framework could potentially achieve that. Whereas most previous work focuses on the static setting (e.g., with images), we postulate that some of the causal factors could be discovered if the learner is allowed to interact with its environment. The agent can experiment with different actions and observe their effects. More specifically, we hypothesize that some of these factors correspond to aspects of the environment which are independently controllable, i.e., that there exists a policy and a learnable feature for each such aspect of the environment, such that this policy can yield changes in that feature with minimal changes to other features that explain the statistical variations in the observed data. We propose a specific objective function to find such factors and verify experimentally that it can indeed disentangle independently controllable aspects of the environment without any extrinsic reward signal.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

Citations in the corpus

Learning deep representations by mutual information estimation and maximization (2018, background): “…(Gretton et al., 2012), independent (Hyv ¨ arinen & Oja, 2000; Hinton, 2002; Dinh et al., 2014; Brakel & Bengio, 2017), disentangled (Schmidhuber, 1992; Rifai et al., 2012; Bengio et al., 2013; Chen et al., 2018; Gonzalez-Garcia et al., 2018), or independently controllable (Thomas et al., 2017).”
Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICA (2021, background): “The assumption that individual actions often affect only one factor of variation has been leveraged for disentanglement by Thomas et al. (2017).”
On the Binding Problem in Artificial Neural Networks (2020, background): “…are disentangled can be approached by optimizing for some notion of (statistical) independence between features (e.g. Schmidhuber, 1992c; Chen et al., 2016; Higgins et al., 2017a), sparse feature updates across time (Whitney, 2016), or independent controllability of features (Thomas et al., 2017).”
Robustly Disentangled Causal Mechanisms: Validating Deep Representations for Interventional Robustness (2018, background): “…effects on encodings may give rise to the development of novel, robustly disentangled representation learning algorithms, for example in the interactive learning environment (Thomas et al., 2017) or when weak forms of supervision are available (Bouchacourt et al., 2017; Locatello et al., 2019).”
Measuring the tendency of CNNs to Learn Surface Statistical Regularities (2017, background): “o generate some other perturbation of the dataset that yields a new generalization gap. With respect to promising new directions to solve the high level abstraction learning problem, recent work like [42,18] aim to learn good disentangled feature representations by combining unsupervised learning an”
Visual Reinforcement Learning with Imagined Goals (2018, background): “Many prior works have also focused on learning controllable and disentangled representations [41, 5, 6, 38, 7, 46].”
Disentangling the independently controllable factors of variation by interacting with the world (2018, methodology): “We build on a previously studied [Thomas et al., 2017] mechanism for representation learning that has close ties to intrinsic motivation mechanisms and causality.”
State Representation Learning for Control: An Overview (2018, methodology): “States can be learned by using the idea that factors such as objects correspond to ‘inde-pendently controllable’ aspects of the world that can be discovered by interacting with the environment [Thomas et al., 2017].”
Inductive biases for deep learning of higher-level cognition (2020, background): “This line of thinking is directly related to the work in machine learning and reinforcement learning on controllability of aspects of the environment (Bengio et al., 2017; Thomas et al., 2017).”
The Profiling Potential of Computer Vision and the Challenge of Computational Empiricism (2019, unknown): “to produce ‘disentangled representations’ [74].”
Task-Agnostic Dynamics Priors for Deep Reinforcement Learning (2019, background): “More recent work has proposed approaches to learn separate models for dynamics and rewards and use it to perform online planning (Zhang et al., 2018a) or learn independently controllable factors in the environment (Thomas et al., 2017).”
On Adversarial Mixup Resynthesis (2019, background): “…goals include learning interpretable representations (Chen et al., 2016; Jang et al., 2016), disentanglement of latent variables (Liu et al., 2017; Thomas et al., 2017) or maximisation of mutual information (Chen et al., 2016; Belghazi et al., 2018; Hjelm et al., 2019) between the input and the…”
Auto-Encoding Total Correlation Explanation (2018, background): “The work of (Thomas et al., 2017; Bengio et al., 2017) is similar in spirit, identifying disentangled factors as changes in a latent embedding that can be controlled via reinforcement learning.”
Composable Planning with Attributes (2018, background): “Recent work, such as (Thomas et al., 2017), demonstrates how this could be done.”
EMI: Exploration with Mutual Information (2018, background): “Thomas et al. (2017) aims to learn the representations that maximize the causal relationship between the distributed policies and the representation of changes in the state.”