Relational inductive bias for physical construction in humans and machines

By Jessica B. Hamrick, Kelsey R. Allen, Victor Bapst, Tina Zhu, Kevin R. McKee, Josh Tenenbaum, Peter W. Battaglia. 2018

122 citations recorded by the source corpus.

While current deep learning systems excel at tasks such as object classification, language processing, and gameplay, few can construct or modify a complex system such as a tower of blocks. We hypothesize that what these systems lack is a "relational inductive bias": a capacity for reasoning about inter-object relations and making choices over a structured description of a scene. To test this hypothesis, we focus on a task that involves gluing pairs of blocks together to stabilize a tower, and quantify how well humans perform. We then introduce a deep reinforcement learning agent which uses object- and relation-centric scene and policy representations and apply it to the task. Our results show that these structured representations allow the agent to outperform both humans and more naive approaches, suggesting that relational inductive bias is an important component in solving structured reasoning problems and for building more intelligent, flexible machines.

Subfields: Artificial Intelligence, Mechanical Engineering, Safety Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Mechanical Engineering

In Safety Research

Citations in the corpus

On the Binding Problem in Artificial Neural Networks (2020, background): “…2018; Li et al., 2020; Stanić et al., 2020), visual question answering (Santoro et al., 2017; Palm et al., 2018), abstract visual reasoning (Andreas, 2019), natural language processing (Tai et al., 2015), physical construction (Hamrick et al., 2018) or multi-agent interactions (Sun et al., 2019).”
Structured agents for physical construction (2019, methodology): “DQN learning algorithm : For discrete action outputs, we use Q-learning implemented as a deep Q network (DQN) from Mnih et al. (2015), with Q-values on the edges, similar to Hamrick et al. (2018).”
Relational inductive biases, deep learning, and graph networks (2018, methodology): “…model-free (Wang et al., 2018b) and model-based (Hamrick et al., 2017; Pascanu et al., 2017; Sanchez-Gonzalez et al., 2018) continuous control, for model-free reinforcement learning (Hamrick et al., 2018; Zambaldi et al., 2018), and for more classical approaches to planning (Toyer et al., 2017).”
Hierarchical Planning for Long-Horizon Manipulation with Geometric and Symbolic Scene Graphs (2020, methodology): “Both the task planning and motion generation models are based off the Graph Neural Networks (GNNs) [13] which are deep learning modules for graph processing.”
Graph neural networks in particle physics (2020, background): “In [86] and [87], a GN is used to guide the policy of a reinforcement learning agent and build graphs that represent physical scenes.”
DOM-Q-NET: Grounded RL on Structured Language (2019, background): “There has been work in robot locomotion that uses graph neural networks (GNNs) to model the physical body (Wang et al., 2018; Hamrick et al., 2018).”
Principal Neighbourhood Aggregation for Graph Nets (2020, methodology): “We present a consistently well-performing and parameter efficient encode-process-decode architecture [23] for GNNs.”
ETA Prediction with Graph Neural Networks in Google Maps (2021, methodology): “We compose 3 GN blocks into an encode-process-decode architecture [11], and learn a separate model for each horizon h.”
The Tools Challenge: Rapid Trial-and-Error Learning in Physical Problem Solving (2019, background): “In line with other work on physical problem solving showing that object-oriented and relational priors are important (Hamrick et al., 2018), we incorporate an object-based prior for sampling actions.”
Combinatorial optimization and reasoning with graph neural networks (2021, methodology): “The power comes from using the aforementioned encode-process-decode framework [Hamrick et al. , 2018].”
Hamiltonian Neural Networks (2019, background): “A variety of previous works have taken this path and produced excellent results [41, 14, 43, 34, 17, 6].”
Neural Execution of Graph Algorithms (2019, methodology): “Learning to execute graph algorithms To define a generic architecture for learning to execute algorithms, we leverage the encode-process-decode paradigm [10].”
Deep reinforcement learning with relational inductive biases (2018, background): “This provides a powerful capacity for combinatorial generalization, where their learned building blocks can be composed to represent and reason about novel scenarios (Battaglia et al., 2016; Raposo et al., 2017; Dai et al., 2017; Wang et al., 2018; Selsam et al., 2018; Hamrick et al., 2018; Kipf et ”
Graph Element Networks: adaptive, structured computation and memory (2019, background): “In general, GNNs are applied in settings where nodes and edges are explicit and match concrete entities of the domain; for instance, nodes can be objects (Chang et al., 2016; Battaglia et al., 2016; Hamrick et al., 2018) or be parts of objects (Wang et al.”
Know Your Action Set: Learning Action Relations for Reinforcement Learning (2022, background): “…with a rich relational structure, such as morphological control (Wang et al., 2018; Sanchez-Gonzalez et al., 2018; Pathak et al., 2019), multi-agent RL (Tacchetti et al., 2019), physical construction (Hamrick et al., 2018), and structured perception in games like StarCraft (Zambaldi et al., 2018).”
Learning to build physical structures better over time (2020, background): “While many tasks in this literature involve passive judgments about physical scenes, a promising new direction is to consider tasks that involve active interventions on physical systems to achieve various goals (Allen, Smith, & Tenenbaum, 2019; Hamrick et al., 2018).”
Learning to simulate and design for structural engineering (2020, background): “Related to structural engineering domain, (Hamrick et al., 2018) uses reinforcement learning to train a graph neural network policy that learns to glue pairs of blocks to stabilize a block tower under gravity in a physics engine.”