Policy Evaluation Networks
By Jean Harb, Tom Schaul, Doina Precup, Pierre-Luc Bacon. 2020
42 citations recorded by the source corpus.
Many reinforcement learning algorithms use value functions to guide the search for better policies. These methods estimate the value of a single policy while generalizing across many states. The core idea of this paper is to flip this convention and estimate the value of many policies, for a single set of states. This approach opens up the possibility of performing direct gradient ascent in policy space without seeing any new data. The main challenge for this approach is finding a way to represent complex policies that facilitates learning and generalization. To address this problem, we introduce a scalable, differentiable fingerprinting mechanism that retains essential policy information in a concise embedding. Our empirical results demonstrate that combining these three elements (learned Policy Evaluation Network, policy fingerprints, gradient ascent) can produce policies that outperform those that generated the training data, in zero-shot manner.
Subfields: Artificial Intelligence, Civil and Structural Engineering
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Matthew Riemer
- Irina Rish
- Khimya Khetarpal
- Doina Precup
- Sergey Levine
- Joelle Pineau
- Eric Eaton
- Chelsea Finn
In Civil and Structural Engineering
Citations in the corpus
- Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “Moreover, another group of recent approaches focuses on learning a meta-critic, which explicitly guides updates to the agent’s policy rather than simply guiding its actions (Harb et al., 2020; Sung, Zhang, Xiang, Hospedales, and Yang, 2017; Xu, Cao, and Chen, 2019).”