Reinforcement Learning With High-Dimensional, Continuous Actions

By I. Baird, Klopf Leemon C.. 1993

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Reinforcement learning in robotics: A survey (2013, background): “Other possible function approximators for value functions include wire fitting, whichBaird and Klopf (1993) suggested as an approach that makes continuous action selection feasible.”
Reinforcement Learning with Factored States and Actions (2004, methodology): “The state or action space could also be continuous (Baird and Klopf, 1993; Santamaria et al., 1998).”
Reinforcement Learning in Continuous Action Spaces (2007, background): “Baird and Klopf propose the Wire Fitting (WF) algorithm, which efficiently stores an approximation of the complete Qfunction [ 11 ].”
SkyAI: Highly modularized reinforcement learning library (2010, methodology): “A generic linear function approximator and wire-fitting [15] are implemented.”
Reinforcement Learning Through Gradient Descent (1999, methodology): “Furthermore, some algorithms, such as VAPS (chapter 6) with φ>0, or Wire Fitting (Baird & Klopf, 1993), already pass the output of y through a similar-looking function, so there is very little additional cost to fold in this new calculation.”
Reinforcement Learning: An Introduction (1998, methodology): “Baird and Klopf (1993) introduced a novel memory-based approach and used it as the function approximation method for Q-learning applied to the pole-balancing task.”
The problem with DDPG: understanding failures in deterministic environments with sparse rewards (2019, methodology): “Issues when combining RL with function approximation have been studied for a long time (Baird & Klopf, 1993; Boyan & Moore, 1995; Tsitsiklis & Van Roy, 1997).”
Conjugate Markov Decision Processes (2011, background): “Wire-fitting (Baird & Klopf, 1993) and Product of Experts (Sallans & Hinton, 2004) are not applicable because the actions are discrete and state continuous, respectively.”
Reinforcement learning in continuous time: advantage updating (1994, methodology): “Another method, wire fitting, is described in Baird and Klopf (1993b).”
Locally Weighted Learning for Control (1997, methodology): “Baird and Klopf (1993) apply nearest neighbor techniques and weighted averaging to reinforcement learning and Thrun (1996) and Thrun and O'Sullivan (1996) apply similar techniques to robot learning.”
Algorithms for Sequential Decision Making (1996, background): “Continuous actions can also be discretized, although speci c algorithms [8] can solve this type of environment more e ectively.”
CAQL: Continuous Action Q-Learning (2019, background): “For instance, wire-ﬁtting (Gaskett et al., 1999; III & Klopf, 1993) approximates Q-values piecewise-linearly over a discrete set of points, chosen to ensure the maximum action is one of the extreme points.”
Kernel-Based Reinforcement Learning (2004, methodology): “Baird and Klopf (1993) apply the nearest neighbor algorithm to reinforcement learning, and Connell and Utgoff (1987), Peng and Williams (1995), and Atkeson, Moore, and Schaal (1997) apply locally weighted regression to physical control problems.”
A comparison of direct and model-based reinforcement learning (1997, background): “A number of researchers have focused on this case (Gullapalli, 1990; Jordan and Jacobs, 1990; Williams, 1990; Millington, 1991; Baird and Klopf, 1993; Bradtke, 1993; Bradtke et al., 1994; Doya, 1996).”