Learning Visual Predictive Models of Physics for Playing Billiards

By Katerina Fragkiadaki, Pulkit Agrawal, Sergey Levine, Jitendra Malik. 2015

282 citations recorded by the source corpus.

The ability to plan and execute goal specific actions in varied, unexpected settings is a central requirement of intelligent agents. In this paper, we explore how an agent can be equipped with an internal model of the dynamics of the external world, and how it can use this model to plan novel actions by running multiple internal simulations ("visual imagination"). Our models directly process raw visual input, and use a novel object-centric prediction formulation based on visual glimpses centered on objects (fixations) to enforce translational invariance of the learned physical laws. The agent gathers training data through random interaction with a collection of different environments, and the resulting model can then be used to plan goal-directed actions in novel environments that the agent has not seen before. We demonstrate that our agent can accurately plan actions for playing a simulated billiards game, which requires pushing a ball into a target position or into collision with another ball.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, AI for Physical Sciences

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In AI for Physical Sciences

Citations in the corpus

CLEVRER: CoLlision Events for Video REpresentation and Reasoning (2019, background): “…related to research on learning scene dynamics for physical and causal reasoning (Lerer et al., 2016; Battaglia et al., 2013; Mottaghi et al., 2016; Fragkiadaki et al., 2016; Battaglia et al., 2016; Chang et al., 2017; Agrawal et al., 2016; Finn et al., 2016; Shao et al., 2014; Fire & Zhu, 2016;…”
Structured agents for physical construction (2019, background): “Modern advances in learning-based approaches have opened new avenues for using vector and convolutional representations for physical reasoning (Wu et al., 2015; 2016; 2017; Mottaghi et al., 2016; Fragkiadaki et al., 2016; Finn et al., 2016; Agrawal et al., 2016; Lerer et al., 2016; Li et al., 2016; ”
Stochastic Prediction of Multi-Agent Interactions from Partial Observations (2019, background): “( Vondrick et al., 2016a) train a conditional CNN to predict future feature vectors, and several papers (e.g., (Lerer et al., 2016; Mottaghi et al., 2016; Wu et al., 2017; van Steenkiste et al., 2018; Fragkiadaki et al., 2016; Watters et al., 2017)) predict future object states.”
Flexible Neural Representation for Physics Prediction (2018, background): “However, future predictors operating directly on 2D image representations often fail to generate sharp object boundaries and struggle with occlusions and remembering objects when they are no longer visually observable [1, 17, 16, 28, 29, 33, 34, 19].”
Relational Neural Expectation Maximization: Unsupervised Discovery of Objects and their Interactions (2018, background): “…state-to-state predictions, often describing the state of a system as a set of compact object-descriptions that are either used as an input to the system (Battaglia et al., 2016; Chang et al., 2016; Fragkiadaki et al., 2015; Grzeszczuk et al., 1998) or for training purposes (Watters et al., 2017).”
Visual Interaction Networks: Learning a Physics Simulator from Video (2017, methodology): “A third class of methods [5, 8, 9, 23], like our Visual Interaction Network, have been used to predict future state descriptions from pixels.”
A Compositional Object-Based Approach to Learning Physical Dynamics (2016, methodology): “Because physics does not change across inertial frames, it sufﬁces to separately predict the future state of each object conditioned on the past states of itself and the other objects in its neighborhood, similar to Fragkiadaki et al. (2015b).”
A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning (2017, background): “While the focus in this work is to define a generative model for high dimensional videos of simple physical systems, several recent works have combined physical models of the world with deep learning to learn the dynamics of objects in more complex but low-dimensional environments [3, 4, 9, 34].”
Combining self-supervised learning and imitation for vision-based rope manipulation (2017, background): “Impressive results have been obtained on playing Atari games [10], a large suite of robotic tasks in simulation [11], [12], [13], [14], [15], [16] and real world robotic manipulation tasks involving rigid objects in controlled settings [17], [18].”
Learning to Poke by Poking: Experiential Learning of Intuitive Physics (2016, background): “Only very recently, [3, 29, 30, 24] addressed the problem of model based control from vision in synthetic domains of manipulating two degree of freedom robotic arm, inverted pendulum, billiards and Atari games.”
Graph networks as learnable physics engines for inference and control (2018, neutral): “One of our key contributions is an approach for learning physical dynamics models (Grzeszczuk et al., 1998; Fragkiadaki et al., 2015; Battaglia et al., 2016; Chang et al., 2016; Wat-3 ters et al., 2017; Ehrhardt et al., 2017; Amos et al., 2018).”
Learning and Using the Arrow of Time (2018, background): “efficient) by the physics engine in [8] with our extension to handle multiple balls.”
Learning model-based planning from scratch (2017, background): “Recently [7] trained a “visual imagination” model to control simulated billiards systems, though their system did not learn to plan.”
Learning to Perform Physics Experiments via Deep Reinforcement Learning (2016, positive): “When the purpose is learning an intuitive understanding of dynamics it is possible to do interesting work with entirely synthetic data (Fragkiadaki et al., 2016; Lerer et al., 2016).”
Physics 101: Learning Physical Object Properties from Unlabeled Videos (2016, background): “Recently, researchers started to tackle concrete scenarios for understanding physics from vision [11, 15, 18, 19, 23, 24, 25, 27], some building on recent advances in deep learning.”
Metacontrol for Adaptive Imagination-Based Optimization (2017, positive): “In other similar work, Fragkiadaki et al. (2015) made use of “visual imaginations” for action planning.”
SE3-nets: Learning rigid body motion using deep neural networks (2016, background): “Physics prediction: Recent work in deep learning has looked at the problem of predicting the stability of stacked blocks [15], ball motion [16], and object dynamics in images [17].”