Fast deep reinforcement learning using online adjustments from the past
By Steven Stenberg Hansen, Pablo Sprechmann, Alexander Pritzel, André Barreto, Charles Blundell. 2018
47 citations recorded by the source corpus.
We propose Ephemeral Value Adjusments (EVA): a means of allowing deep reinforcement learning agents to rapidly adapt to experience in their replay buffer. EVA shifts the value predicted by a neural network with an estimate of the value function found by prioritised sweeping over experience tuples from the replay buffer near the current state. EVA combines a number of recent ideas around combining episodic memory-like structures into reinforcement learning agents: slot-based storage, content-based retrieval, and memory-based planning. We show that EVA is performant on a demonstration task and Atari games.
Subfields: Artificial Intelligence, Management Science and Operations Research, Reinforcement Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Yoshua Bengio
- Pieter Abbeel
- Yuxi Li
- Caiming Xiong
- Richard Socher
- Ruslan Salakhutdinov
- Quoc V. Le
In Management Science and Operations Research
- Julien Pérolat
- Marc Lanctot
- Karl Tuyls
- Edward Hughes
- Daniel Hennes
- Jean-Baptiste Lespiau
- Joel Z. Leibo
- Shayegan Omidshafiei
In Reinforcement Learning
- Karl Tuyls
- Yuxi Li
- Sergey Levine
- Joel Z. Leibo
- Julien Pérolat
- Marc Lanctot
- Pieter Abbeel
- Jean-Baptiste Lespiau
Citations in the corpus
- OpenSpiel: A Framework for Reinforcement Learning in Games (2019, methodology): “Traditional Single-Agent RL Algorithms We currently have three algorithms usable for traditional (single-agent) RL: Deep Q-Networks (DQN) [44], Advantage Actor-Critic (A2C) [42], and Ephemeral Value Adjustments (EVA) [22].”
- From Discriminant to Complete: Reinforcement Searching-Agent Learning for Weakly Supervised Object Detection (2020, methodology): “…works, we will explore the potential of using the saliency detection technique [1], [58], [59] to obtain the better local discriminative regions and apply the more advanced reinforcement learning algorithms [60], [61] to further improve the learning performance of the proposed WSOD framework.”
- Sample-Efficient Deep Reinforcement Learning via Episodic Backward Update (2018, background): “Ephemeral Value Adjustments (EVA) [7] combines the values of two separate networks, where one is the standard DQN and another is a trajectory-based value network.”
- Generalization of Reinforcement Learners with Working and Episodic Memory (2019, background): “Episodic memory has been shown to help reinforcement learning agents adapt more quickly and thereby boost data efficiency (Blundell et al., 2016; Pritzel et al., 2017; Hansen et al., 2018).”