Batch Reinforcement Learning

By Sascha Lange, Thomas Gabel, Martin A. Riedmiller. 2012

699 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Reinforcement Learning

Citations in the corpus

Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems (2020, background): “While this was arguably less of an issue when reinforcement learning methods utilized low-dimensional or linear parameterizations, and therefore relied on small datasets for small problems that were easy to collect or simulate (Lange et al., 2012), once deep networks are incorporated into reinforcem”
Offline Reinforcement Learning with Implicit Q-Learning (2021, background): “Ofﬂine reinforcement learning (RL) addresses the problem of learning effective policies entirely from previously collected data, without online interaction (Fujimoto et al., 2019; Lange et al., 2012).”
Accelerating Online Reinforcement Learning with Offline Datasets (2020, methodology): “due to error accumulation with distribution shift [8, 21]. Ofﬂine RL methods aim to address this issue, often by constraining the actor updates to avoid excessive deviation from the data distribution [23, 41, 13, 14, 12, 21, 8, 22, 38, 26, 47]. One class of these methods utilize importance sampling ”
Towards Causal Representation Learning (2021, background): “off-policy learning is considered, in particular in the batch (or observational) setting [146], issues of causality become subtle [164, 81].”
Causality for Machine Learning (2019, background): “n that it sometimes effectively directly estimates do-probabilities (on-policy learning). However, as soon as off-policy learning is considered, in particular in the batch (or observational) setting (Lange et al., 2012), issues of causality become subtle (Lu et al., 2018; Gottesman et al., 2018). Re”
Off-Policy Deep Reinforcement Learning without Exploration (2018, background): “Most modern off-policy deep reinforcement learning algorithms fall into the category of growing batch learning (Lange et al., 2012), in which data is collected and stored into an experience replay dataset (Lin, 1992), which is used to train the agent before further data collection occurs.”
Reinforcement Learning in Healthcare: A Survey (2019, background): “efficient learning, and are the major driving force behind the proposal of modern Batch RL (BRL) [39].”
MOPO: Model-based Offline Policy Optimization (2020, methodology): “To overcome some of the sample efﬁciency challenges, we study the batch ofﬂine RL setting [40].”
MOReL : Model-Based Offline Reinforcement Learning (2020, background): “[12], and has applications in healthcare [33, 34, 35], recommendation systems [36, 37, 38, 39], dialogue systems [40, 19, 41], and autonomous driving [42].”
D4RL: Datasets for Deep Data-Driven Reinforcement Learning (2020, methodology): “The oﬄine RL setting [Lange et al., 2012], also referred to as full batch RL, where agents must learn from large, ﬁxed datasets of logged interaction, provides a bridge between the ﬂexi-ble control formalism provided by RL and the data-driven success of supervised learning.”
Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learning (2020, background): “Batch reinforcement learning (RL) (Ernst et al., 2005; Lange et al., 2011) is the problem of learning a policy from a fixed, previously recorded, dataset without the opportunity to collect new data through interaction with the environment.”
A Framework for Data-Driven Robotics (2019, background): “Batch RL: Batch RL describes the scenario where the learning experience is fixed a priori [62].”
Behavior Regularized Offline Reinforcement Learning (2019, background): “Oﬄine RL (also known as batch RL (Lange et al., 2012)) considers the problem of learning a policy π from a ﬁxed dataset D consisting of single-step transitions ( s, a, r, s (cid:48) ) .”
Safe Policy Improvement with Soft Baseline Bootstrapping (2019, background): “This approach, also known as Batch Reinforcement Learning (Lange et al., 2012), is motivated by the many real-world applications that naturally fit a setting where data collection and optimization are decoupled (contrary to online learning which integrates the two): e.g. dialogue systems (Singh et a”
Striving for Simplicity in Off-policy Deep Reinforcement Learning (2019, methodology): “Most of these algorithms fall into the category of growing batch learning [34], in which data is collected and stored into an experience replay buffer [37], which is used to train the agent before further data collection.”
Safe Policy Improvement with Baseline Bootstrapping (2017, methodology): “However, in a growing batch batch setting (Lange et al., 2012), it might be valuable to keep exploring the way Πb-SPIBB does.”
Safe Policy Improvement with an Estimated Baseline Policy (2019, methodology): “This setting is called batch RL (Lange, Gabel, and Riedmiller 2012).”
A Minimalist Approach to Offline Reinforcement Learning (2021, methodology): “In offline RL (historically known as batch RL), the agent learns from a fixed-sized dataset, collected by some arbitrary and possibly unknown process [Lange et al., 2012].”