Propagation Networks for Model-Based Control Under Partial Observation

By Yunzhu Li, Jiajun Wu, Jun-Yan Zhu, Josh Tenenbaum, Antonio Torralba, Russ Tedrake. 2018

158 citations recorded by the source corpus.

There has been an increasing interest in learning dynamics simulators for model-based control. Compared with off-the-shelf physics engines, a learnable simulator can quickly adapt to unseen objects, scenes, and tasks. However, existing models like interaction networks only work for fully observable systems; they also only consider pairwise interactions within a single time step, both restricting their use in practical systems. We introduce Propagation Networks (PropNet), a differentiable, learnable dynamics model that handles partially observable scenarios and enables instantaneous propagation of signals beyond pairwise interactions. With these innovations, our propagation networks not only outperform current learnable physics engines in forward simulation, but also achieves superior performance on various control tasks. Compared with existing deep reinforcement learning algorithms, model-based control with propagation networks is more accurate, efficient, and generalizable to novel, partially observable scenes and tasks.

Subfields: Computational Theory and Mathematics, Control and Systems Engineering, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Control and Systems Engineering

In Statistical and Nonlinear Physics

Citations in the corpus

Learning to Simulate Complex Physics with Graph Networks (2020, background): “, “interaction networks”, can learn to simulate rigid body, mass-spring, n-body, and robotic control systems (Battaglia et al., 2016; Chang et al., 2016; SanchezGonzalez et al., 2018; Mrowca et al., 2018; Li et al., 2019; Sanchez-Gonzalez et al., 2019), as well as non-physical systems, such as multi”
CLEVRER: CoLlision Events for Video REpresentation and Reasoning (2019, methodology): “We apply the Propagation Network (PropNet) (Li et al., 2019) for dynamics modeling, which is a learnable physics engine that, extending from the interaction networks (Battaglia et al.”
SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation (2020, background): “Given additional task-speciﬁc information about the connectivity among the particles, alternative architectures such as graph neural networks may be better at capturing the structure of the particles [50].”
Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids (2018, background): “Li et al. (2018) proposed learning to propagate signals along the interaction graph.”
Visual Grounding of Learned Physical Models (2020, methodology): “Recently, the DPI-Net (Li et al., 2019a) has achieved strong results in modeling a variety of rigid and deformable object dynamics.”
Self-Supervised Learning of State Estimation for Manipulating Deformable Linear Objects (2019, background): “More related to our work, Li et al. [18] and Battaglia et al. [19] model ropes as mass-spring systems, and use graph networks to learn rope dynamics.”
Grounding Physical Concepts of Objects and Events Through Dynamic Visual Reasoning (2021, methodology): “Finally, we train a dynamics prediction network (Li et al., 2019b) based on the reﬁned object trajectories and optimize it jointly with other learning parts in this uniﬁed framework.”