Experience Replay for Real-Time Reinforcement Learning Control

By Sander Adam, Lucian Busoniu, Robert Babuška. 2012

260 citations recorded by the source corpus.

Subfields: Computational Theory and Mathematics, Electrical and Electronic Engineering, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Electrical and Electronic Engineering

In Management Science and Operations Research

Citations in the corpus

Designing Neural Network Architectures using Reinforcement Learning (2016, background): “Finally, when the state space is large or exploration is costly, the experience replay technique (Lin, 1993) has proved useful in experimental settings (Adam et al., 2012; Mnih et al., 2015).”
Online Contrastive Divergence with Generative Replay: Experience Replay without Storing Data (2016, background): “In ER, the data acquired during the online learning process is stored explicitly and presented repeatedly to the online learning algorithm, such as reinforcement learning (RL) (Adam et al, 2012), deep reinforcement learning (DRL) (Mnih et al, 2015), or supervised learning (McClelland et al, 1995).”
Replay in Deep Learning: Current Approaches and Missing Biological Elements (2021, neutral): “Experience replay has also been widely used in reinforcement learning (Mnih et al., 2015, 2013; Van Hasselt, Guez, & Silver, 2016; Lillicrap et al., 2016; Lin, 1992; Adam, Busoniu, & Babuska, 2011; Foerster et al., 2017; Kapturowski, Ostrovski, Dabney, Quan, & Munos, 2019; Atkinson et al., 2021).”
Deep Exploration via Randomized Value Functions (2017, background): “In particular, the agent may learn to make more effective decisions within fewer episodes (Adam et al., 2012).”
A novel DDPG method with prioritized experience replay (2017, background): “experience replay mechanism and an experience is a four-element tuple (st ,at ,rt ,st+1)) [7], [8] and exploits target network approaches to stabilize the training process.”
The importance of experience replay database composition in deep reinforcement learning (2015, background): “Having an off-policy algorithm helps to some extent since it allows for the use of experience replay [14, 21, 22].”
Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systems (2014, methodology): “…and use available data more effectively, the experience replay technique has been proposed in the context of RL for discrete-time systems (Adam, Busoniu, & Babuska, 2012; Dung, Komeda, & Takagi, 2008; Kalyanakrishnan & Stone, 2007; Lin, 1992; Wawrzynski, 2009; Xu, Jagannathan, & Lewis,…”
Multiobjective Reinforcement Learning: A Comprehensive Overview (2015, background): “As discussed in [103], although value function approximators can be integrated with HRL,”
A Deeper Look at Planning as Learning from Replay (2015, methodology): “Another example are model-free methods that store observed samples for reuse at a later time, like LSTD (Bradtke & Barto, 1996; Boyan, 2002) or experience replay (Lin, 1992; Adam et al., 2012).”
An empirical investigation of the challenges of real-world reinforcement learning (2020, methodology): “One approach is to take existing algorithms and validate their feasibility to run in real-time [Adam et al. 2011]. Another approach is to design algorithms with the explicit goal of running in real-time [Cai et al. 2017; Wang and Yuan 2015]. Recently Ramstedt and Pal [2019] presented a different vie”
Challenges of real-world reinforcement learning: definitions, benchmarks and analysis (2021, background): “One approach is to take existing algorithms and validate their feasibility to run in real-time (Adam et al. 2011).”
Online Model Learning Algorithms for Actor-Critic Control (2015, methodology): “To overcome this problem, several techniques have been proposed to remember and reuse measured data, such as experience replay (Adam et al., 2011; Lin, 1992; Wawrzyński, 2009) and prioritised sweeping (Moore and Atkeson, 1993).”