Stable Fitted Reinforcement Learning

By Geoffrey J. Gordon. 1995

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

An Emphatic Approach to the Problem of Off-policy Temporal-Difference Learning (2015, background): “An Emphatic Approach to Off-policy TD Learning Over the years, several different approaches have been taken to solving the problem of off-policy learning.”
An Introduction to Deep Reinforcement Learning (2018, unknown): “This effect is due to the mappings used in the value iteration algorithms with bootstrapping (e.g., Equation 4.2 for the Q-learning algorithm) that propagate errors more strongly with a high discount factor.”
Recent advances in reinforcement learning in finance (2021, methodology): “Fitted 𝑄-learning (Gordon, 1996) is a generalization of the classical 𝑄-learning algorithm with functional approximations and it is applied in an off-line setting with a precollected dataset in the form of tuples (𝑠, 𝑎, 𝑟, 𝑠′) with 𝑠′ ∼ 𝑃(𝑠, 𝑎) and 𝑟 = 𝑟(𝑠, 𝑎), which is random.”
Reinforcement Learning in Continuous Time and Space (2000, methodology): “Efforts have been made to eliminate some of these difficulties by using appropriate function approximators (Gordon, 1996; Sutton, 1996; Tsitsiklis & Van Roy, 1997), adaptive state partitioning and aggregation methods (Moore, 1994; Singh, Jaakkola, & Jordan, 1995; Asada, Noda, & Hosoda, 1996; Pareigi”
What are the Statistical Limits of Offline RL with Linear Function Approximation? (2020, background): “See [Gordon, 1995, 1996, Munos and Moore, 1999, Ormoneit and Sen, 2002, Kakade, 2003, Zanette et al., 2019] for papers on this topic and additional assumptions that mitigate this issue.”
Reinforcement Learning Through Gradient Descent (1999, methodology): “For POMDPs, however, even lookup tables are not guaranteed to converge with existing algorithms (Gordon, 96).”
Instabilities of Offline RL with Pre-Trained Neural Representation (2021, unknown): “See [Gordon, 1995, 1996, Munos and Moore, 1999, Ormoneit and Sen, 2002, Kakade, 2003, Zanette et al., 2019] for discussion on this topic.”
Reinforcement Learning: An Introduction (1998, methodology): “4 Graybiel (2000) is a brief primer on the basal ganglia.”