Dynamics-aware Embeddings

By William F. Whitney, Rajat Agarwal, Kyunghyun Cho, Abhinav Kumar Gupta. 2019

59 citations recorded by the source corpus.

In this paper we consider self-supervised representation learning to improve sample efficiency in reinforcement learning (RL). We propose a forward prediction objective for simultaneously learning embeddings of states and action sequences. These embeddings capture the structure of the environment's dynamics, enabling efficient policy learning. We demonstrate that our action embeddings alone improve the sample efficiency and peak performance of model-free RL on control from low-dimensional states. By combining state and action embeddings, we achieve efficient learning of high-quality policies on goal-conditioned continuous control from pixel observations in only 1-2 million environment steps.

Subfields: Artificial Intelligence, Management Science and Operations Research, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Karl Pertsch
2. Joseph J. Lim
3. Youngwoon Lee

In Management Science and Operations Research

1. Karl Pertsch
2. Joseph J. Lim
3. Youngwoon Lee

In Statistical and Nonlinear Physics

1. Karl Pertsch
2. Joseph J. Lim
3. Youngwoon Lee

Citations in the corpus

• Accelerating Reinforcement Learning with Learned Skill Priors (2020, background): “Using fixed-length skills allows for scalable skill learning and has proven to be effective in prior works [7, 8, 27, 31, 28, 37].”