Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learning

By Nan Rosemary Ke, Aniket Rajiv Didolkar, Sarthak Mittal, Anirudh Goyal, Guillaume Lajoie, Stefan Bauer, Danilo Jimenez Rezende, Yoshua Bengio, Michael C. Mozer, C. Pal. 2021

77 citations recorded by the source corpus.

Inducing causal relationships from observations is a classic problem in machine learning. Most work in causality starts from the premise that the causal variables themselves are observed. However, for AI agents such as robots trying to make sense of their environment, the only observables are low-level variables like pixels in images. To generalize well, an agent must induce high-level variables, particularly those which are causal or are affected by causal variables. A central goal for AI and causality is thus the joint discovery of abstract representations and causal structure. However, we note that existing environments for studying causal induction are poorly suited for this objective because they have complicated task-specific causal graphs which are impossible to manipulate parametrically (e.g., number of nodes, sparsity, causal chain length, etc.). In this work, our goal is to facilitate research in learning representations of high-level variables as well as causal structures among them. In order to systematically probe the ability of methods to identify these variables and structures, we design a suite of benchmarking RL environments. We evaluate various representation learning algorithms from the literature and find that explicitly incorporating structure and modularity in models can help causal induction in model-based reinforcement learning.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Reinforcement Learning

Citations in the corpus

Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICA (2021, background): “…learning probabilistic or causal graphs between high-level latent variables extracted from low-level observations (Bengio, 2019; Schölkopf, 2019; Schölkopf et al., 2021; Goyal and Bengio, 2021; Ke et al., 2021), there have been comparatively few practical solutions contributed to the literature.”
Inductive biases for deep learning of higher-level cognition (2020, background): “knowledge is generic, defined over abstract variables, and can be applied on different instances factorizing knowledge in terms of abstract variables and functions that encapsulate how these variables interact with each other [99,114,115]”
Neural Production Systems (2021, methodology): “The physics environment (Ke et al., 2021) simulates a simple physical world.”
D’ya Like DAGs? A Survey on Structure Learning and Causal Discovery (2021, background): “…reviews with a focus on time-series causal inference and discovery [51]; frameworks for dynamical SCMs with ODEs [163, 188]; guides on the foundations for causal discovery [48]; some example applications [123, 146, 149, 211, 284]; textbooks on causal inference and causal discovery [184, 190, 228].”