Batch reinforcement learning in a complex domain

By Shivaram Kalyanakrishnan, Peter Stone. 2007

85 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Management Science and Operations Research

Citations in the corpus

Online Contrastive Divergence with Generative Replay: Experience Replay without Storing Data (2016, methodology): “Since ER uses recorded data in chunks, it has sometimes been deemed a batch learning approach (Kalyanakrishnan and Stone, 2007).”
Striving for Simplicity in Off-policy Deep Reinforcement Learning (2019, background): “[31] Shivaram Kalyanakrishnan and Peter Stone.”
Adaptive treatment of anemia on hemodialysis patients: A reinforcement learning approach (2011, background): “That is, each experience is used to update the agent’s action-value function immediately, and is then discarded [25].”
Batch Reinforcement Learning Through Continuation Method (2021, background): “Off-policy reinforcement learning has been extensively studied [11, 20, 30, 23, 31], with many works [12, 21, 2] focusing on variants of Q-learning.”
Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systems (2014, methodology): “…replay technique has been proposed in the context of RL for discrete-time systems (Adam, Busoniu, & Babuska, 2012; Dung, Komeda, & Takagi, 2008; Kalyanakrishnan & Stone, 2007; Lin, 1992; Wawrzynski, 2009; Xu, Jagannathan, & Lewis, 2012), without providing a proof of convergence and stability.”
Experience Replay for Real-Time Reinforcement Learning Control (2012, methodology): “Later on, Kalyanakrishnan and Stone [16] illustrated using simulations that reusing data in Q-learning leads to a performance similar to that of batch-RL algorithms, and Wawrzynski [37] showed that ER can be added to actor–critic”
Swarm robotics: a review from the swarm engineering perspective (2013, methodology): “In this work, the authors applied neural networks as function approximators together with fast learning algorithms (Kalyanakrishnan and Stone 2007 ).”
The Effects of Memory Replay in Reinforcement Learning (2017, methodology): “The performance of RL with experience replay is similar to the batch RL but in an incremental way Lagoudakis & Parr (2003); Kalyanakrishnan & Stone (2007); Ernst et al. (2005).”
Batch Reinforcement Learning (2012, methodology): “…in the whole range of being as close to the pure batch approach as using only two iterations (Riedmiller et al, 2008) to recalculating the policy after every few interactions—e.g. after finishing one episode in a shortest-path problem (Kalyanakrishnan and Stone, 2007; Lange and Riedmiller, 2010a).”
Adaptive Control of Epileptic Seizures using Reinforcement Learning (2010, methodology): “Also, the algorithm has been shown to make efficient use of training data [34], which is especially important in medical applications, where data may be sparse and expensive to collect.”
Adaptive Treatment of Epilepsy via Batch-mode Reinforcement Learning (2008, methodology): “Also, the algorithm has been shown to make efﬁcient use of training data (Kalyanakrishnan & Stone 2007), which is especially important in medical applications, where data may be sparse and expensive to collect.”
Fitted Q-iteration in continuous action-space MDPs (2007, methodology): “Nevertheless, FQI has been applied successfully in a number of challenging domains and was found to perform well even when used with non-averager function approximation methods, such as neural networks [12, 13].”
Convergent Fitted Value Iteration with Linear Function Approximation (2011, background): “FVI has become increasingly popular especially in the field of “batch-mode Reinforcement Learning” [13, 7] where a policy is learned from a fixed batch of data that was collected by a prior agent.”
Overcoming Model Bias for Robust Offline Deep Reinforcement Learning (2020, methodology): “Early pure batch algorithms have been mainly used to solve rather simple MDPs and / or contain the implicit assumption that the batch contains a balanced set of transition samples spanning the entire state-action space (Kalyanakrishnan and Stone 2007).”