Structured agents for physical construction

By Victor Bapst, Alvaro Sanchez-Gonzalez, Carl Doersch, Kimberly L. Stachenfeld, Pushmeet Kohli, Peter W. Battaglia, Jessica B. Hamrick. 2019

107 citations recorded by the source corpus.

Physical construction---the ability to compose objects, subject to physical dynamics, to serve some function---is fundamental to human intelligence. We introduce a suite of challenging physical construction tasks inspired by how children play with blocks, such as matching a target configuration, stacking blocks to connect objects together, and creating shelter-like structures over target objects. We examine how a range of deep reinforcement learning agents fare on these challenges, and introduce several new approaches which provide superior performance. Our results show that agents which use structured representations (e.g., objects and scene graphs) and structured policies (e.g., object-centric actions) outperform those which use less structured representations, and generalize better beyond their training when asked to reason about larger scenes. Model-based agents which use Monte-Carlo Tree Search also outperform strictly model-free agents in our most challenging construction problems. We conclude that approaches which combine structured representations and reasoning with powerful learning are a key path toward agents that possess rich intuitive physics, scene understanding, and planning.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Robotics & Control

Citations in the corpus

Toward Causal Representation Learning (2021, background): “Note that the problem of object-centric representation learning [11], [40], [84], [87], [88], [138], [155], [160], [255], [262] can also be considered a special case of disentangled factorization as discussed here.”
Towards Causal Representation Learning (2021, background): “learning [10, 39, 83, 86, 87, 138, 155, 160, 262, 255] can also be considered a special case of disentangled factorization as discussed here.”
CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning (2020, methodology): “It is worth noting that similar setups were proposed previously in a less realistic setting as in (Janner et al., 2018; Bapst et al., 2019; McCarthy et al.; Akkaya et al., 2019; Fahlman, 1974; Winston, 1970; Winograd, 1972).”
Emergent Tool Use From Multi-Agent Autocurricula (2019, methodology): “In [55], an agent solves construction tasks in a 2-D environment using both model-based and model-free methods.”
Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers (2020, background): “, for rigid-body physics in the context of human reasoning [5, 64, 3] or weather predictions [51], particles are also a popular basis for fluid flow problems [33, 61, 48].”
Learning Object-Centric Representations of Multi-Object Scenes from Multiple Views (2021, background): “As a result, “multi-object” or object-centric representations have emerged as a promising approach to scene understanding, improving sample-efﬁciency and generalization for many down-stream applications like relational reasoning and control [5, 20, 3].”
Planning with Diffusion for Flexible Behavior Synthesis (2022, background): “Because learned dynamics models are normally meant to be proxies for environment dynamics, improvements are often achieved by structuring the model according to the underlying causal process (Bapst et al., 2019).”
Graph neural networks in particle physics (2020, background): “In [86] and [87], a GN is used to guide the policy of a reinforcement learning agent and build graphs that represent physical scenes.”
Rapid trial-and-error learning with simulation supports flexible tool use and physical reasoning (2019, methodology): “…that test artificial agents’ abilities to predict and reason using physics, such as the concurrently developed PHYRE benchmark (Bakhtin, van der Maaten, Johnson, Gustafson, & Girshick, 2019) and others (Bapst et al., 2019; Ge, Lee, Renz, & Zhang, 2016; Wenke, Saunders, Qiu, & Fleming, 2019).”
Using Unity to Help Solve Intelligence (2020, background): “• Structured agents for physical construction (Bapst et al., 2019): This paper uses a continuous, procedurally-generated 2D world that tests an agent’s ability to solve a variety of physics-based construction tasks.”
Unsupervised Object Keypoint Learning using Local Spatial Predictability (2020, background): “, 2018) or value function (Bapst et al., 2019) as GNNs for RL tasks on continuous control or physical construction domains.”
Learning to build physical structures better over time (2020, methodology): “Our specific approach draws most direct inspiration from prior work in developmental science (Cortesa et al., 2018; Bullock & Lütkenhaus, 1988) and AI (Bapst et al., 2019; Jones, Hager, & Khudanpur, 2019) that have examined physical construction behavior.”