Practical Reinforcement Learning in Continuous Spaces

By William D. Smart, Leslie Pack Kaelbling. 2000

271 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Toward Interpretable-AI Policies Using Evolutionary Nonlinear Decision Trees for Discrete-Action Systems (2020, methodology): “Consider the MountainCar control system problem (see the supplementary material for details), which has been extensively studied using various AI methods [12]–[14].”
Safe Exploration of State and Action Spaces in Reinforcement Learning (2012, methodology): “Smart and Kaelbling (2000) also use examples, training runs to bootstrap the Q-learning approach for their HEDGER algorithm.”
Learning from observation using primitives (2001, methodology): “[111], Smart and Kaelbling [118], and Forbes and Andre [44] give some insight on various methods that can be used to encode the Q value in a function approximator.”
Effective reinforcement learning for mobile robots (2002, methodology): “In previous work [5], [6], we presented an algorithm, Hedger, that addresses the problems associated with value-function approximation.”
A comprehensive survey on safe reinforcement learning (2015, methodology): “The gaussian exploration also introduces a random component in the algorithms proposed by García and Fernández (2012) and Smart and Kaelbling (2000).”
From model-based control to data-driven control: Survey, classification and perspective (2013, methodology): “The main idea of [122] was to use an existing approximated Q-function to train neighboring Q-functions and to use a hyper-elliptic hull to prevent extrapolation.”
Autonomous blimp control using model-free reinforcement learning in a continuous state and action space (2007, background): “Continuous state spaces have also been considered by Smart and Kaelbling [17].”
Extending Q-Learning to General Adaptive Multi-Agent Systems (2003, background): “Several recent function approximation schemes [11, 12] enable Q-learning to work well in continuous spaces.”
Tree-Based Batch Mode Reinforcement Learning (2005, background): “The bang-bang nature was also observed in Smart and Kaelbling (2000), where continuous and a discrete action spaces are treated on the “Car on the Hill” problem, with qualitatively the same results.”
Mo' States Mo' Problems: Emergency Stop Mechanisms from Observation (2019, background): “Smart and Kaelbling [22] bootstrap Q-values from expert state-action demonstrations.”
On Kernelized Multi-armed Bandits (2017, background): “…in a variety of sequential decision making problems, such as dynamic pricing in economics (Besbes and Zeevi, 2009), reinforcement learning with continuous state/action spaces (Kaelbling et al., 1996; Smart and Kaelbling, 2000), and power control in wireless communication (Chiang et al., 2008).”
Planning in Discrete and Continuous Markov Decision Processes by Probabilistic Programming (2015, unknown): “More sophisticate approaches are possible [5,26].”
Graph kernels and Gaussian processes for relational reinforcement learning (2006, background): “Future work will investigate how reinforcement techniques such as local linear models [23] and the use of convex hulls to make safe predictions [25] can be applied in RRL.”
Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models (2015, methodology): “Many of the pioneering algorithms in this area, such as R − Max [7] and E(3) [8], achieve efficient exploration that scales polynomially with the number of parameters in the agent’s state space (see also [9, 10]).”
Exploration and apprenticeship learning in reinforcement learning (2005, methodology): “Schaal and Atkeson (1994) and Smart and Kaelbling (2000) both give examples where learning is signiﬁcantly faster when bootstrapping from a teacher.”
Reinforcement Learning: A Tutorial Survey and Recent Advances (2009, unknown): “TD(λ) can also be used in SARSA (see SARSA(λ) of Sutton (1996)) especially when the learning is episodic.”
CAQL: Continuous Action Q-Learning (2019, methodology): “…Bellman residual loss using global nonlinear optimizers, such as the cross-entropy method (CEM) for QT-Opt (Kalash-nikov et al., 2018), gradient ascent (GA) for actor-expert (Lim et al., 2018), and action discretization (Uther & Veloso, 1998; Smart & Kaelbling, 2000; Lazaric et al., 2008).”
Integrating Guidance into Relational Reinforcement Learning (2004, methodology): “There is, however, an approach which has been followed with success, and which consists of guiding the Q-learner with examples of " reasonable " strategies, provided by a teacher (Smart and Kaelbling, 2000).”