Unsupervised Learning for Physical Interaction through Video Prediction

By Chelsea Finn, Ian J. Goodfellow, Sergey Levine. 2016

1,116 citations recorded by the source corpus.

A core challenge for an agent learning to interact with the world is to predict how its actions affect objects in its environment. Many existing methods for learning the dynamics of physical interactions require labeled object information. However, to scale real-world interaction learning to a variety of scenes and objects, acquiring labeled data becomes increasingly impractical. To learn about physical object motion without labels, we develop an action-conditioned video prediction model that explicitly models pixel motion, by predicting a distribution over pixel motion from previous frames. Because our model explicitly predicts motion, it is partially invariant to object appearance, enabling it to generalize to previously unseen objects. To explore video prediction for real-world interactive agents, we also introduce a dataset of 59,000 robot interactions involving pushing motions, including a test set with novel objects. In this dataset, accurate prediction of videos conditioned on the robot's future actions amounts to learning a "visual imagination" of different futures based on different courses of action. Our experiments show that our proposed method produces more accurate video predictions both quantitatively and qualitatively, when compared to prior methods.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Reinforcement Learning

Citations in the corpus

Scaling Autoregressive Video Models (2019, methodology): “Finally, we tried sampling temperature 0.9 and 1.0 only on the BAIR Robot Pushing validation set and found that temperature 0.9 consistently gave more robust predictions and better results on all extrinsic metrics.”
Neural Discrete Representation Learning (2017, background): “Recent advances in generative modelling of images [38, 12, 13, 22, 10], audio [37, 26] and videos [20, 11] have yielded impressive samples and applications [24, 18].”
Video Pixel Networks (2016, background): “s based on deep neural networks (Srivastava et al., 2015a; Oh et al., 2015), to models that predict quantized image patches (Ranzato et al., 2014), incorporate motion priors (Patraucean et al., 2015; Finn et al., 2016) or use adversarial losses (Mathieu et al., 2015; Vondrick et al., 2016). Despite ”
VideoGPT: Video Generation using VQ-VAE and Transformers (2021, methodology): “Plenty of methods have been proposed for video prediction on the BAIR Robot dataset (Finn et al., 2016; Ebert et al., 2017; Babaeizadeh et al., 2017; Denton et al., 2017; Denton & Fergus, 2018; Lee et al., 2018) where the future frames are predicted given the past frame(s) and (or) action(s) of a…”
Generating Videos with Scene Dynamics (2016, background): “Our work is also related to efforts to predict the future in video [33, 22, 43, 50, 42, 17, 8, 54] as well as concurrent work in future generation [6, 15, 20, 49, 55].”
MERLOT: Multimodal Neural Script Knowledge Models (2021, methodology): “Some modeling choices include using pixels [28, 100], graphs [10], euclidean distance using sensors [2], or studying cycle consistency across time Epstein et al.”
Value Prediction Network (2017, background): “Building an accurate observationprediction model is often very challenging when the observation space is large [23, 5, 13, 4] (e.”
Parallel Multiscale Autoregressive Density Estimation (2017, background): “We evaluate our model on ImageNet, Caltech-UCSD Birds (CUB), the MPII Human Pose dataset (MPII), the Microsoft Common Objects in Context dataset (MS-COCO), and the Google Robot Pushing dataset.”
NIPS 2016 Tutorial: Generative Adversarial Networks (2016, methodology): “t. The agent can query the model with dierent potential actions and choose actions that the model predicts are likely to yield a desired state of the world. For a recent example of such a model, see Finn et al. (2016b), and for a recent example of the use of such a model for planning, see Finn and L”
Photographic Image Synthesis with Cascaded Refinement Networks (2017, background): “[9] explicitly model pixel motion and also condition on action.”
Dynamic Filter Networks (2016, neutral): “[2] recently independently proposed a mechanism called (convolutional) dynamic neural advection that is very similar to ours.”
CLEVRER: CoLlision Events for Video REpresentation and Reasoning (2019, background): “…2016; Battaglia et al., 2013; Mottaghi et al., 2016; Fragkiadaki et al., 2016; Battaglia et al., 2016; Chang et al., 2017; Agrawal et al., 2016; Finn et al., 2016; Shao et al., 2014; Fire & Zhu, 2016; Pearl, 2009; Ye et al., 2018), either directly from images (Finn et al., 2016; Ebert et al.,…”
Structured agents for physical construction (2019, background): “Modern advances in learning-based approaches have opened new avenues for using vector and convolutional representations for physical reasoning (Wu et al., 2015; 2016; 2017; Mottaghi et al., 2016; Fragkiadaki et al., 2016; Finn et al., 2016; Agrawal et al., 2016; Lerer et al., 2016; Li et al., 2016; ”
Flexible Neural Representation for Physics Prediction (2018, background): “However, future predictors operating directly on 2D image representations often fail to generate sharp object boundaries and struggle with occlusions and remembering objects when they are no longer visually observable [1, 17, 16, 28, 29, 33, 34, 19].”
A Compositional Object-Based Approach to Learning Physical Dynamics (2016, background): “…observations to physical judgments (Lerer et al., 2016; Li et al., 2016; Mottaghi et al., 2015; 2016) or passive (Lerer et al., 2016; Srivastava et al., 2015; Sutskever et al., 2009) and action-conditioned (Agrawal et al., 2016; Finn et al., 2016; Fragkiadaki et al., 2015b) motion prediction.”
The limits and potentials of deep learning for robotics (2018, background): “…grounded in real world experience, are well suited for closed-loop control, and can be learned and Recent approaches to end-to-end training of deep networks forgo the use of explicit physics models, learning predictive models and controls from raw experiences [9], [16], [18], [36], [129], [132].”
RoboNet: Large-Scale Multi-Robot Learning (2019, methodology): “8k), Fetch (5k) [6], GoogleRobot (56k) [5]”
Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control (2018, methodology): “In visual MPC, we use a transformation-based video prediction architecture, ﬁrst proposed by Finn et al. [31].”
Composable Action-Conditioned Predictors: Flexible Off-Policy Learning for Robot Navigation (2018, background): “Prediction has been an active area of research in computer vision, including prediction of future images [9, 10, 11], motion [12], and even physical events [13, 14, 15].”