Gradient Descent for General Reinforcement Learning

By Leemon C. Baird III, Andrew W. Moore. 1998

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning in neural networks: An overview (2014, neutral): “Many variants of traditional RL exist (e.g., Abounadi, Bertsekas, & Borkar, 2002; Baird, 1995; Baird & Moore, 1999; Barto, Sutton, & Anderson, 1983; Bertsekas, 2001; Bradtke, Barto, & Kaelbling, 1996; Brafman & Tennenholtz, 2002; Kaelbling, Littman, & Cassandra, 1995; Lagoudakis & Parr, 2003; Maei…”
Multiagent learning using a variable learning rate (2002, background): “There are single-agent learning techniques that are capable of playing stochastic policies [1,2,16,30].”
Infinite-Horizon Policy-Gradient Estimation (2001, background): “Roughly speaking, the variance of GPOMDP grows with 1 / (1 − β ) , while the bias decreases as a function of 1 / (1 − β ) . ing both the theoretical results of the present paper on a toy problem, and practical aspects of the algorithms on a number of more realistic problems.”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “The parameters or weights of C are trained to maximize reward by a standard RL method such as Q-learning or similar methods [17, 292, 293, 172, 254, 212, 262, 10, 122, 188, 157, 281, 26, 216, 197, 272, 311, 9, 163, 174, 22, 27, 2, 137, 276, 156, 284].”
Learning from Scarce Experience (2002, background): “We show positive empirical results and provide the sample complexity bound.”
A New Softmax Operator for Reinforcement Learning (2016, unknown): “The values and policy never converge.”
Reinforcement Learning with Factored States and Actions (2004, unknown): “See for example Baird and Moore (1999).”
Evolutionary Bilevel Optimization for Complex Control Tasks (2015, background): “Recently, much progress has been made in using policy search methods to solve such control tasks [4, 5, 6].”
Learning to trade via direct reinforcement (2001, background): “” When Q-learning is extended to function approximators, it has been shown in many cases that there are simple Markov decision processes for which the algorithms fail to converge [64].”
Adaptive Tile Coding for Value Function Approximation (2007, background): “Even for reinforcement learning algorithms with guaranteed convergence ( Baird & Moore 1999; Lagoudakis & Parr 2003), achieving high performance in practice requires finding an appropriate representation for the function approximator.”
Policy Gradient Methods for Robot Control (2003, methodology): “For these reasons, the traditional greedy policy improvement based approaches seem to be less likely to scale for continuous and high dimensional domains[1].”
Evolutionary Function Approximation for Reinforcement Learning (2006, methodology): “Beielstein and Markon (2002) use a similar approach to develop tests for determining which individuals should survive.”
Reinforcement learning in robotic applications: a comprehensive survey (2021, background): “Thus, the small change in value function leads to less oscillatory behavior of policy (Baird III and Moore 1999).”
Reinforcement Learning in POMDP's via Direct Gradient Ascent (2000, unknown): “The “VAPS” algorithm described in Baird and Moore (1999) showed how the algorithms in Marbach and Tsitsiklis (1998) could be interpreted as combining both value-function and policy-gradient approaches.”