Independently Controllable Features

By Emmanuel Bengio, Valentin Thomas, Joelle Pineau, Doina Precup, Yoshua Bengio. 2017

67 citations recorded by the source corpus.

Finding features that disentangle the different causes of variation in real data is a difficult task, that has nonetheless received considerable attention in static domains like natural images. Interactive environments, in which an agent can deliberately take actions, offer an opportunity to tackle this task better, because the agent can experiment with different actions and observe their effects. We introduce the idea that in interactive environments, latent factors that control the variation in observed data can be identified by figuring out what the agent can control. We propose a naive method to find factors that explain or measure the effect of the actions of a learner, and test it in illustrative experiments.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Learning deep representations by mutual information estimation and maximization (2018, background): “Usefulness of a representation is not just a matter of information content: representational characteristics like independence also play an important role (Gretton et al., 2012; Hyvärinen & Oja, 2000; Hinton, 2002; Schmidhuber, 1992; Bengio et al., 2013; Thomas et al., 2017).”
Toward Causal Representation Learning (2021, background): “This could be done by ensuring that they are invariant across problems, exhibit sparse changes to actions or that they can be independently intervened upon [22], [30], [217].”
Towards Causal Representation Learning (2021, background): “An emerging line of work devoted to the intersection of RL and causality includes [13, 21, 164, 37, 50, 275, 1].”
Causality for Machine Learning (2019, background): “Recent work devoted to the field between RL and causality includes (Bareinboim et al., 2015; Bengio et al., 2017; Lu et al., 2018; Buesing et al., 2018; Dasgupta et al., 2019; Zhang and Bareinboim, 2019).”
Automatically Composing Representation Transformations as a Means for Generalization (2018, background): “Just as the motivation behind disentangled representations [11, 17, 49, 82, 85] is to uncover the latent factors of variation, the motivation behind disentangled programs is to uncover the latent organization of a task.”
Inductive biases for deep learning of higher-level cognition (2020, background): “This line of thinking is directly related to the work in machine learning and reinforcement learning on controllability of aspects of the environment (Bengio et al., 2017; Thomas et al., 2017).”
EMI: Exploration with Mutual Information (2018, methodology): “Furthermore, the method uses f-divergence (Nowozin et al.,2016) estimation of Jensen-Shannon divergence rather than the KL divergence to estimate the mutual information for better numerical stability.Bengio et al. (2017);Thomas et al.(2017a;b) deﬁne selectivity, which lower bounds the conditional mu”
Interpreting neural computations by examining intrinsic and embedding dimensionality of neural activity (2021, background): “In these situations, the computational constraint that likely determines the embedding properties of latent variables is their controllability [70]; i.”
Interventional Robustness of Deep Latent Variable Models (2018, background): “…notion of interventional effects on encodings also gives rise to the development of novel, robustly disentangled representation learning algorithms, for example in the interactive learning environment (Thomas et al., 2017) or when weak forms of supervision are available (Bouchacourt et al., 2017).”
Feature Control as Intrinsic Motivation for Hierarchical Reinforcement Learning (2017, methodology): “introduced the notion of feature selectivity that measures how much a feature can be controlled, independently from other features [6].”
Auto-Encoding Total Correlation Explanation (2018, background): “The work of (Thomas et al., 2017; Bengio et al., 2017) is similar in spirit, identifying disentangled factors as changes in a latent embedding that can be controlled via reinforcement learning.”
Model-free, Model-based, and General Intelligence (2018, background): “Should the moving objects emerge from a general learning scheme, from prior knowledge, or from a combination of both [Mugan and Kuipers, 2012; Thomas et al., 2017; Marcus, 2018a]?”
Unsupervised Visuomotor Control through Distributional Planning Networks (2019, background): “Related to learning goal representations, a number of prior works have considered the problem of learning control-centric state representations from interaction data [38, 5, 22, 30, 45, 29], for use with planning or reinforcement learning”