Reinforcement Learning Through Gradient Descent

By Leemon C. Baird III. 1999

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Increasing the Action Gap: New Operators for Reinforcement Learning (2015, methodology): “The method of advantage learning was proposed by Baird (1999) as a means of increasing the gap between the optimal and suboptimal actions in the context of residual algorithms applied to continuous time problems.”
Munchausen Reinforcement Learning (2020, methodology): “Taking the limit τ → 0, we retrieve Advantage Learning (AL) [5, 7] (see Appx.”
Policy Search using Paired Comparisons (2003, methodology): “The VAPS method (Baird, 1999) follows policy and state-action value gradients simul- STRENS AND MOORE taneously.”
Gradient temporal-difference learning algorithms (2011, methodology): “Previous work on gradient methods for TD minimized this objective fn (Baird 1995, 1999) Mean Square Projected Bellman Error (MSPBE) But if you minimize the expected TD error: , then you get the solution Even in the tabular case (no FA) Backward-bootstrapping example (1) (Dayan 1992) Clearly, the…”
Learning to Cooperate via Policy Search (2000, methodology): “Williams introduced the notion of policy search for reinforcement learning in his REINFORCE algorithm [19, 20], which was generalized to a broader class of error criteria by Baird and Moore [2, 3].”
Fast gradient-descent methods for temporal-difference learning with linear function approximation (2009, unknown): “In this paper we explore the development of true stochastic gradient-descent algorithms for temporal-difference learning with linear function approximation, building on the work of Baird (1995; 1999) and of Sutton, Szepesv´ ari and Maei (2009).”
Reinforcement learning for control: Performance, stability, and deep approximators (2018, methodology): “These include the Semi-Markov Decision Process (SMDP) framework (Bradtke and Duff, 1995), which adds the time it takes to transition between states to the MDP framework, as well as the advantage learning algorithm (Baird, 1999) and the consistent Bellman operator (Bellemare et al.”
Fast Gradient-Descent Methods for Temporal-Difference Learning with Linear Function Approximation (2009, unknown): “To illustrate this, we use the standard counterexample proposed by Baird (1999).”
Direct Policy Search using Paired Statistical Tests (2001, methodology): “Harmon & Baird (1996) address some of these issues, and apply residual advantage learning to a similar pursuer-evader game.”
Reinforcement Learning: An Introduction (1998, methodology): “Baird (1995, 1999) extended it t o TDL based on stochastic gradient desc ent, and Engel, Mann or, and Meir (2003) exten ded it to lea st squares (O (n2)) method s known as Gaussian Pro cess TDL.”