Evolved Policy Gradients

By Rein Houthooft, Richard Y. Chen, Phillip Isola, Bradly C. Stadie, Filip Wolski, Jonathan Ho, Pieter Abbeel. 2018

238 citations recorded by the source corpus.

We propose a metalearning approach for learning gradient-based reinforcement learning (RL) algorithms. The idea is to evolve a differentiable loss function, such that an agent, which optimizes its policy to minimize this loss, will achieve high rewards. The loss is parametrized via temporal convolutions over the agent's experience. Because this loss is highly flexible in its ability to take into account the agent's history, it enables fast task learning. Empirical results show that our evolved policy gradient algorithm (EPG) achieves faster learning on several randomized environments compared to an off-the-shelf policy gradient method. We also demonstrate that EPG's learned loss can generalize to out-of-distribution test time tasks, and exhibits qualitatively different behavior from other popular metalearning algorithms.

Subfields: Artificial Intelligence, Reinforcement Learning, Evolutionary Computation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Sergey Levine
  2. Yoshua Bengio
  3. Chelsea Finn
  4. Pieter Abbeel
  5. Quoc V. Le
  6. Ruslan Salakhutdinov
  7. Mohit Bansal
  8. Dacheng Tao

In Reinforcement Learning

  1. Sergey Levine
  2. Pieter Abbeel
  3. Chelsea Finn
  4. Shimon Whiteson
  5. Abhishek Gupta
  6. Ofir Nachum
  7. Yuxi Li
  8. Shixiang Shane Gu

In Evolutionary Computation

  1. Shimon Whiteson
  2. Sergey Levine
  3. Chelsea Finn
  4. Jian Peng
  5. Chen Liang
  6. J. Urgen Schmidhuber
  7. Satinder Singh
  8. Yoshua Bengio

Citations in the corpus