Real-time reinforcement learning by sequential Actor-Critics and experience replay

By Pawel Wawrzynski. 2009

116 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Molecular Biology, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Molecular Biology

In Reinforcement Learning

Citations in the corpus

Continuous control with deep reinforcement learning (2015, methodology): “The frames were downsampled to 64x64 pixels and the 8-bit RGB values were converted to floating point scaled to [0, 1].”
High-Dimensional Continuous Control Using Generalized Advantage Estimation (2015, methodology): “While the formula has been proposed in prior work [8, 19], our analysis is novel and enables GAE to be applied with a more general set of algorithms, including the batch trust-region algorithm we use for our experiments.”
IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures (2018, methodology): “Related previous work on off-policy RL include (Precup et al., 2000; 2001; Wawrzynski, 2009; Geist & Scherrer, 2014) and (Harutyunyan et al., 2016).”
Accelerating Online Reinforcement Learning with Offline Datasets (2020, background): “Off-policy RL algorithms are designed to reuse off-policy data during training, and have been studied extensively [21, 6, 27, 13, 9, 5, 34, 52, 48, 4].”
DeepMind Control Suite (2018, background): “A running planar biped based on (Wawrzyński, 2009).”
EPOpt: Learning Robust Neural Network Policies Using Model Ensembles (2016, methodology): “We evaluated the proposed EPOpt- algorithm on the 2D hopper (Erez et al., 2011) and halfcheetah (Wawrzynski, 2009) benchmarks using the MuJoCo physics simulator (Todorov et al., 2012).1 Both tasks involve complex second order dynamics and direct torque control.”
Learning to Play in a Day: Faster Deep Reinforcement Learning by Optimality Tightening (2016, methodology): “A technique called “experience replay” (Lin, 1992; Wawrzynski, 2009) encourages this property and quickly emerged as a standard step in the well-known deep Q-learning framework (Mnih et al.”
The importance of experience replay database composition in deep reinforcement learning (2015, background): “Having an off-policy algorithm helps to some extent since it allows for the use of experience replay [14, 21, 22].”
Experience Replay for Real-Time Reinforcement Learning Control (2012, methodology): “Later on, [16] illustrated using simulations that reusing data in Q-learning leads to a performance similar to that of batch RL algorithms, and [37] showed that ER can be added to actor-critic RL algorithms without endangering their convergence properties.”
Online Model Learning Algorithms for Actor-Critic Control (2015, methodology): “To overcome this problem, several techniques have been proposed to remember and reuse measured data, such as experience replay (Adam et al., 2011; Lin, 1992; Wawrzyński, 2009) and prioritised sweeping (Moore and Atkeson, 1993).”
Autonomous reinforcement learning with experience replay. (2013, methodology): “In this paper, an actor–critic reinforcement learning algorithm was introduced that combines experience replay (Wawrzyński, 2009) with the fixed point method of step-size estimation, improved upon its version presented in Wawrzyński (2010).”
Learning state representation for deep actor-critic control (2016, methodology): “In [3] this technique was combined with the actor-critic approach to form the Deep Deterministic Policy Gradient (DDPG) algorithm, which was able to solve several continuous control tasks, including the cart-pole benchmark [4] and the “cheetah” locomotion task introduced in [5].”
dm_control: Software and Tasks for Continuous Control (2020, background): “A running planar biped based on (Wawrzyński, 2009).”