Online actor critic algorithm to solve the continuous-time infinite horizon optimal control problem

By Kyriakos G. Vamvoudakis, Frank L. Lewis. 2009 International Joint Conference on Neural Networks, 2009

1,607 citations recorded by the source corpus.

Subfields: Computational Theory and Mathematics, Control and Systems Engineering, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/894e0bb5b42294f6b89b4f5b9498fd86139c96f9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Control and Systems Engineering

In Electrical and Electronic Engineering

Citations in the corpus

Decentralized Stabilization for a Class of Continuous-Time Nonlinear Interconnected Systems Using Online Learning Optimal Control Approach (2014, methodology): “Abu-Khalaf and Lewis [41] derived an offline optimal control scheme for nonlinear systems with saturating actuators.”
A novel actor-critic-identifier architecture for approximate optimal control of uncertain nonlinear systems (2013, methodology): “The actor update law in (19) also differs in the sense that the update law in Vamvoudakis and Lewis (2010) is purely motivated by the stability analysis whereas the proposed actor update law is based on the minimization of the Bellman error with an additional term for stability.”
Online Model Learning Algorithms for Actor-Critic Control (2015, background): “Continuous-time variants of actor-critic algorithms (Hanselmann et al., 2007; Vamvoudakis and Lewis, 2010) and multi-agent actor-critic schemes (Li et al., 2008; Pennesi and Paschalidis, 2010) are not considered here.”
Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systems (2014, background): “…for solving the HJB equation for continuous-time (CT) nonlinear systems either require complete knowledge of the system dynamics (Doya, 2000; Vamvoudakis & Lewis, 2010) or lack a rigorous stability analysis (Doya, 2000; Murray et al., 2002), except for Bhasin et al. (2012); Vamvoudakis et…”
Closed‐form solution to finite‐horizon suboptimal control of nonlinear systems (2015, methodology): “Reference [10] proposes an online neural network training scheme for solving the HJB equation corresponding to infinite-horizon problems.”
Adaptive Critic Nonlinear Robust Control: A Survey (2017, methodology): “This motivates us to develop an approximate strategy to overcome the difficulty [66], [68], [69], [76], [77], [81]–[85], [120], [126], [131], which results in the ADP-based neural control design.”
$ {H}_{ {\infty }}$ Tracking Control of Completely Unknown Continuous-Time Systems via Off-Policy Reinforcement Learning (2015, background): “During the last few years, reinforcement learning (RL) [10]–[13] has been extensively used to solve the optimal H2 [14]–[25] and H∞ [26]–[37] regulation problems, and has been successfully applied to several real-world applications [38]–[43].”
Multi-agent differential graphical games (2011, methodology): “RL methods have been applied to learn online in real-time the solutions for optimal control problems for dynamic systems and differential games in [6], [12], [33], [34].”
Multi-player non-zero-sum games: Online adaptive learning solution of coupled Hamilton-Jacobi equations (2011, methodology): “…according to the standard Lyapunov extension theorem the analysis above demonstrates that the state and the weights are UUB (Khalil, 1996; Lewis et al., 1999 Now consider the error dynamics and the output as in Vamvoudakis and Lewis (2010) and assume ¯ σ 3 and ¯ σ 4 are persistently exciting.”