Reinforcement learning is direct adaptive optimal control

By Richard S. Sutton, Andrew Barto, Ronald J. Williams. 1991

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Model-based reinforcement learning with parametrized physical models and optimism-driven exploration (2015, background): “Reinforcement learning (RL) [18], [27] tackles control problems with nonlinear dynamics in a more general framework, which can be either model-based or model-free.”
Reinforcement learning in robotics: A survey (2013, background): “A concise discussion of viewing reinforcement learning as “adaptive optimal control” is presented in (Sutton et al., 1991).”
Truncating Temporal Diierences: on the Eecient Implementation of Td for Reinforcement Learning (1995, background): “The abstract formulation and generality of the reinforcement learning paradigm make it widely applicable, especially in such domains as game-playing (Tesauro, 1992), automatic control (Sutton et al., 1991), and robotics (Lin, 1993).”
Estimator Variance in Reinforcement Learning: Theoretical Problems and Practical Solutions (1997, background): “RL has been characterised as an form of asynchronous, on-line form of Dynamic Programming (DP) (Watkins 1989; Sutton, Barto, & Williams 1992 ).”
Handbook of Learning and Approximate Dynamic Programming (IEEE Press Series on Computational Intelligence) (2004, background): “Thorndike’s law of effect has been rephrased in [5], [56] to offer the following definition of reinforcement-learning: “If an action taken by a learning system is followed by a satisfactory state of affairs, then the tendency of the system to produce that particular action is strengthened or reinfor”
A comparison of direct and model-based reinforcement learning (1997, background): “Direct reinforcement learning algorithms learn a policy or value function without explicitly representing a model of the controlled system (Sutton et al., 1992).”
Vulnerability of Deep Reinforcement Learning to Policy Induction Attacks (2017, background): “The applications of RL range from combinatorial search problems such as learning to play games [2] to autonomous navigation [3], multi-agent systems [4], and optimal control [5].”
Agent Inspired Trading Using Recurrent Reinforcement Learning and LSTM Neural Networks (2017, background): “[1] The idea of this learning system may be viewed as Adaptive Optimal Control, nowadays we call it reinforcement learning [2].”
Nonlinear control systems - A brief overview of historical and recent advances (2017, background): “[87] have argued that reinforcement learning based adaptive optimal schemes are direct adaptive controllers.”
Learning to Act Using Real-Time Dynamic Programming (1995, background): “However, Williams and Baird 1911 have made a valuable contribution to this theory by addressing DP algorithms that are asynchronous at a grain finer than that of either asychronous DP or Q-Learning.”