Learning predictive models of a depth camera & manipulator from raw execution traces

By Byron Boots, Arunkumar Byravan, Dieter Fox. 2014

44 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Biophysics, Computer Vision and Pattern Recognition, Control and Systems Engineering, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biophysics

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Reinforcement Learning

Citations in the corpus

From Pixels to Torques: Policy Learning with Deep Dynamical Models (2015, methodology): “ then use optimal control methods to solve RL problems. Unlike these methods, our approach does not need to estimate value functions and scales to high-dimensional problems. Similar to our approach, (Boots et al., 2014; Levine et al., 2015; van Hoof et al., 2015) recently proposed modelbased RL meth”
Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control (2018, background): “Prior work on action-conditioned video prediction has studied predicting synthetic video game images [16], [29], 3D point clouds [19], and real-world images [30], [31], [32], using both direct autoregressive frame prediction [31], [32], [33] and latent variable models [34], [35].”
Deep visual foresight for planning robot motion (2016, background): “[25] learn a predictive model of RGBD images of a robot arm moving in free space.”
Unsupervised Learning for Physical Interaction through Video Prediction (2016, methodology): “predict future images of a robot arm using nonparametric kernel-based methods [4].”
Learning Visual Predictive Models of Physics for Playing Billiards (2015, methodology): “(Boots et al., 2014) proposes model for predicting the future visual appearance of a robotic arm, but the method is only shown to work when the same object in the same visual environment is considered.”
Stochastic Variational Video Prediction (2017, methodology): “One particularly common assumption is that the environment is deterministic and that there is only one possible future (Chiappa et al., 2017; Srivastava et al., 2015; Boots et al., 2014; Lotter et al., 2017).”
Self-Supervised Visual Planning with Temporal Skip Connections (2017, background): “Action-conditioned video prediction has been explored in the context of synthetic video game images [12, 13] and robotic manipulation [14, 15, 16], and video prediction without actions has been studied for unstructured videos [17, 18, 19] and driving [20, 3].”
SE3-nets: Learning rigid body motion using deep neural networks (2016, methodology): “[11] learn a model using Predictive State Representations to predict raw depth images given a history of prior images and control.”
Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generation (2019, background): “Video prediction and planning have also shown promise in enabling robots to complete a diverse set of visuomotor tasks while generalizing to novel objects (Finn & Levine, 2016; Kalchbrenner et al., 2016; Boots et al., 2014; Byravan & Fox, 2016).”
SE3-Pose-Nets: Structured Deep Dynamics Models for Visuomotor Control (2018, methodology): “[2], our model learns to predict point clouds 2018 IEEE International Conference on Robotics and Automation (ICRA) May 21-25, 2018, Brisbane, Australia”
SE3-Pose-Nets: Structured Deep Dynamics Models for Visuomotor Planning and Control (2017, unknown): “Recent work on deep learning has looked at learning similar predictive models directly in the space of observations, relating changes in pixels or 3D points directly to the applied actions [2]–[4].”
Deep Haptic Model Predictive Control for Robot-Assisted Dressing (2017, background): “[21] learned a predictive model that generates RGB-D images of a robot arm moving in free space.”
Consistent Generative Query Networks (2018, background): “Video Prediction A common limitation of video prediction models is their need for determinism in the underlying environments (Lotter et al., 2017; Srivastava et al., 2015; Boots et al., 2014; Finn et al., 2016; Liu et al., 2017).”
Planning with Goal-Conditioned Policies (2019, background): “When these models are specified manually, planning can be very powerful, but learning such models presents major obstacles: in complex environments with high-dimensional observations such as images, direct prediction of future observations presents a very difficult modeling problem [4, 43, 36, 6, 27”
VideoFlow: A Conditional Flow-Based Model for Stochastic Video Generation (2019, background): “Early work on prediction of future video frames focused on deterministic predictive models (Ranzato et al., 2014; Srivastava et al., 2015; Vondrick et al., 2015; Xingjian et al., 2015; Boots et al., 2014).”
The Nonparametric Kernel Bayes Smoother (2016, methodology): “Examples include the nonparametric kernel Bayes’ filter (nKB-filter) on discrete-time time-invariant state space models [10] and the Semiparametric kernel Monte-Carlo filter (KMC-filter) [16], as well as methods for dynamical system modeling and reinforcement learning applications [14, 21, 23, 4, 3,”
VideoFlow: A Flow-Based Generative Model for Video (2019, background): “…of stochastic prediction, focusing specifically on the case of conditional video prediction: synthesizing raw RGB video frames conditioned on a short context of past observations (Ranzato et al., 2014; Srivastava et al., 2015; Vondrick et al., 2015; Xingjian et al., 2015; Boots et al., 2014).”