Graph networks as learnable physics engines for inference and control

By Alvaro Sanchez-Gonzalez, Nicolas Manfred Otto Heess, Jost Tobias Springenberg, Josh Merel, Martin A. Riedmiller, Raia Hadsell, Peter W. Battaglia. 2018

677 citations recorded by the source corpus.

Understanding and interacting with everyday physical scenes requires rich knowledge about the structure of the world, represented either implicitly in a value or policy function, or explicitly in a transition model. Here we introduce a new class of learnable models--based on graph networks--which implement an inductive bias for object- and relation-centric representations of complex, dynamical systems. Our results show that as a forward model, our approach supports accurate predictions from real and simulated data, and surprisingly strong and efficient generalization, across eight distinct physical systems which we varied parametrically and structurally. We also found that our inference model can perform system identification. Our models are also differentiable, and support online planning via gradient-based trajectory optimization, as well as offline policy optimization. Our framework offers new opportunities for harnessing and exploiting rich knowledge about the world, and takes a key step toward building machines with more human-like representations of the world.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Graph Learning, AI for Physical Sciences

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Graph Learning

In AI for Physical Sciences

Citations in the corpus

Learning to Simulate Complex Physics with Graph Networks (2020, background): “…networks”, can learn to simulate rigid body, mass-spring, n-body, and robotic control systems (Battaglia et al., 2016; Chang et al., 2016; Sanchez-Gonzalez et al., 2018; Mrowca et al., 2018; Li et al., 2019; Sanchez-Gonzalez et al., 2019), as well as non-physical systems, such as…”
Structured agents for physical construction (2019, methodology): “, 2018) and gradient-based optimization via differentiable world models (Sanchez-Gonzalez et al., 2018).”
Stochastic Prediction of Multi-Agent Interactions from Partial Observations (2019, background): “tured and stochastic RNNs. There are several papers that combine standard RNNs (recurrent neural networks) with graph-structured representations, such as Chang et al. (2017); Battaglia et al. (2016); Sanchez-Gonzalez et al. (2018). There are also several papers that extend standard RNNs by adding st”
Relational inductive biases, deep learning, and graph networks (2018, methodology): “This allows a GN’s output to be passed to other deep learning building blocks such as MLPs, CNNs, and RNNs.”
Differentiable Physics-informed Graph Networks (2019, methodology): “Learning physical dynamics A class of models (Grzeszczuk et al., 1998; Battaglia et al., 2016; Chang et al., 2017; Watters et al., 2017; Sanchez-Gonzalez et al., 2018; Kipf et al., 2018) have been proposed based on the assumption that neural networks can learn complex physical interactions and…”
Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data (2019, methodology): “spherical convolutional neural networks (CNNs) for the physical fields on unstructured grid [21], graph networks for object- and relationcentric representations of complex, dynamical systems [22], learning linear embeddings of nonlinear dynamics based on Koopman operator theory [23].”
Isometric Transformation Invariant and Equivariant Graph Convolutional Networks (2020, background): “Several related studies, including those by Sanchez-Gonzalez et al. (2018; 2019); Alet et al. (2019) were focused on applying GNNs to learn physical simulations.”
Recurrent Event Network for Reasoning over Temporal Knowledge Graphs (2019, background): “GN [19, 2] and RRN [17] update node representations by a message passing scheme between time stamps.”
Learning Self-Correctable Policies and Value Functions from Demonstrations with Negative Sampling (2019, neutral): “We refer to [Kurutach et al., 2018, Clavera et al., 2018, Sun et al., 2018b, Chua et al., 2018, Sanchez-Gonzalez et al., 2018, Pascanu et al., 2017, Khansari-Zadeh and Billard, 2011, Luo et al., 2018] and the reference therein for recent work on model-based RL. Off-policy reinforcement learning…”
Contrastive Multi-View Representation Learning on Graphs (2020, background): “, 2019), physical processes (Sanchez-Gonzalez et al., 2018), social net-”
Graph-based knowledge tracing: Modeling student proficiency using graph neural networks (2019, background): “ing for the existing machine learning approaches, the considerable expressive power of graphs has increased research on the GNN, and in recent years, various generalization frameworks and important operations have been developed [2, 7, 23] with successful results in various research areas, such as s”
Learning to simulate and design for structural engineering (2020, background): “Graph neural networks (GNNs) have shown great successes in many learning tasks that require graph data and are applied to many new scientific domains including physics systems (Kipf et al., 2018; Sanchez-Gonzalez et al., 2018; Watters et al., 2017), fluid dynamics(Sanchez-Gonzalez et al.”
Physics-aware Difference Graph Networks for Sparsely-Observed Dynamics (2020, methodology): “, Navier–Stokes equations), we adopt recurrent graph networks (RGN) (Sanchez-Gonzalez et al., 2018) with a graph state Gh as input to combine the spatial”
Graph Convolutional Networks with EigenPooling (2019, methodology): “There are some graph neural networks models designed to reason the dynamics of physical systems where the model is applied to predict future states of nodes given their previous states [1, 32].”
Differentiable Scene Graphs (2019, methodology): “GNNs have also been applied to visual understanding in [11, 38, 36] and control [34, 2].”
Know Your Action Set: Learning Action Relations for Reinforcement Learning (2022, background): “…networks (Battaglia et al., 2018) have been explored in RL tasks with a rich relational structure, such as morphological control (Wang et al., 2018; Sanchez-Gonzalez et al., 2018; Pathak et al., 2019), multi-agent RL (Tacchetti et al., 2019), physical construction (Hamrick et al., 2018), and…”