Reinforcement learning and adaptive dynamic programming for feedback control

By Frank L. Lewis, Draguna L. Vrabie. 2009

1,337 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Reinforcement Learning, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Reinforcement Learning

In Robotics & Control

Citations in the corpus

Memory-based control with recurrent neural networks (2015, neutral): “Similar ideas had previously been exploited in NFQCA [4] and in the ADP [13] community.”
Decentralized Stabilization for a Class of Continuous-Time Nonlinear Interconnected Systems Using Online Learning Optimal Control Approach (2014, positive): “In light of [13], the ADP technique is closely related to reinforcement learning when engaging in the research of feedback control.”
Policy Iteration Adaptive Dynamic Programming Algorithm for Discrete-Time Nonlinear Systems (2014, methodology): “Only in [36] and [37] a brief sketch of policy iteration algorithm for discrete-time systems was displayed, whereas the stability and convergence properties were not discussed.”
Online adaptive optimal control for continuous-time Markov jump linear systems using a novel policy iteration algorithm (2015, neutral): “From the viewpoint of control engineering, RL algorithms can be viewed as a class of adaptive controllers which solve the optimal control problem based on reward information which gives information on the performance of a given controller [37][42].”
Safe Reinforcement Learning Using Robust MPC (2019, background): “Some attempts at combining RL and the linear quadratic regulator have been presented in [14], [15].”
Cooperative Adaptive Cruise Control: A Reinforcement Learning Approach (2011, neutral): “control systems community [23].”
Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics (2012, background): “Indeed, it can be viewed that we solve the following matrix H at each iteration step k Once this matrix is obtained, the control policy can be updated by Bradtke et al. (1994) and Lewis and Vrabie (2009).”
Reinforcement Learning and Feedback Control: Using Natural Decision Methods to Design Optimal Adaptive Controllers (2012, neutral): “Many surveys of ADP are available [20]–[24].”
Reinforcement learning for control: Performance, stability, and deep approximators (2018, background): “In the LQR case, the optimal Q-function is: This can be rewritten in the following quadratic form (Lewis and Vrabie, 2009): By solving then ∂ Q ∗ ( x k , u k ) /∂ u k = 0, we get: which is in fact (24).”
Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systems (2014, neutral): “A survey of RL-based feedback control designs is found in Lewis and Vrabie (2009), Lewis, Vrabie, and Vamvoudakis (2012).”
Reinforcement Learning for Partially Observable Dynamic Processes: Adaptive Dynamic Programming Using Measured Output Data (2011, methodology): “Recent surveys are given by Lewis and Vrabie [19] and Wang et al. [38].”
Adaptive Dynamic Programming for Control: A Survey and Recent Advances (2021, background): “It is noted that most of the existing ADP literature is biased toward discrete-time systems [62], [63], [69], [101], [102].”
Value Iteration Adaptive Dynamic Programming for Optimal Control of Discrete-Time Nonlinear Systems (2016, background): “In recent years, value iteration algorithms have attracted more and more researchers [57]–[64].”