Learning Real-World Robot Policies by Dreaming

By A. Piergiovanni, Alan Wu, Michael S. Ryoo. 2018

33 citations recorded by the source corpus.

Learning to control robots directly based on images is a primary challenge in robotics. However, many existing reinforcement learning approaches require iteratively obtaining millions of robot samples to learn a policy, which can take significant time. In this paper, we focus on learning a realistic world model capturing the dynamics of scene changes conditioned on robot actions. Our dreaming model can emulate samples equivalent to a sequence of images from the actual environment, technically by learning an action-conditioned future representation/scene regressor. This allows the agent to learn action policies (i.e., visuomotor policies) by interacting with the dreaming model rather than the real-world. We experimentally confirm that our dreaming model enables robot learning of policies that transfer to the real-world.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Control and Systems Engineering, Reinforcement Learning, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Reinforcement Learning

In Robotics & Control

Citations in the corpus

Model-Based Reinforcement Learning for Atari (2019, methodology): “Though most of such works do not use image observations, several recent works have incorporated images into real-world (Finn et al., 2016; Finn & Levine, 2016; Babaeizadeh et al., 2017; Ebert et al., 2017; Piergiovanni et al., 2018; Paxton et al., 2018; Rybkin et al., 2018; Ebert et al., 2018) and s”
Dream to Control: Learning Behaviors by Latent Imagination (2019, background): “Piergiovanni et al. (2018) learn robot policies by imagination with a latent dynamics model.”
Emergence of exploratory look-around behaviors through active observation completion (2019, methodology): “We also plan to consider other tasks for transfer such as target-driven navigation (47) and model-based RL (48, 49), where a preliminary exploratory stage is crucial for performing well on downstream tasks.”
Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability (2024, unknown): “Such properties can be useful for representation learning [35, 45, 88, 105, 131], model-based reinforcement learning [39, 40, 42, 43, 94, 96, 98], as well as model-predictive control [27, 30, 35, 41, 92, 104, 142].”