A novel actor-critic-identifier architecture for approximate optimal control of uncertain nonlinear systems

By Shubhendu Bhasin, Rushikesh Kamalapurkar, Marcus Johnson, Kyriakos G. Vamvoudakis, Frank L. Lewis, Warren E. Dixon. at - Automatisierungstechnik, 2013

582 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/a6567a8f08fb793cd4212de1bf2794bb135ac9d0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Derong Liu
2. Qinglai Wei
3. Ding Wang
4. Haibo He
5. Hamidreza Modares
6. Biao Luo
7. Frank L. Lewis
8. Bo Zhao

In Computational Theory and Mathematics

1. Derong Liu
2. Ding Wang
3. Qinglai Wei
4. Hamidreza Modares
5. Haibo He
6. Biao Luo
7. Frank L. Lewis
8. Bo Zhao

In Control and Systems Engineering

1. Derong Liu
2. Ding Wang
3. Hamidreza Modares
4. Qinglai Wei
5. Haibo He
6. Frank L. Lewis
7. Biao Luo
8. Xiong Yang

Citations in the corpus

• Adaptive Dynamic Programming for Control: A Survey and Recent Advances (2021, background): “…by [141] where β 1 > 0 is the learning rate, θ = φ 1 ( x )( f + g ˆ u ) , and (cid:14) An actor NN learning algorithm based on gradient is given by [141] where β 2 and β 3 are positive constants, 1 + θ T θ is introduced for normalization purpose, and the second term on the right-hand side of (33)…”
• Adaptive Critic Nonlinear Robust Control: A Survey (2017, methodology): “This motivates us to develop an approximate strategy to overcome the difficulty [66], [68], [69], [76], [77], [81]–[85], [120], [126], [131], which results in the ADP-based neural control design.”
• Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systems (2014, methodology): “…solving the HJB equation for continuous-time (CT) nonlinear systems either require complete knowledge of the system dynamics (Doya, 2000; Vamvoudakis & Lewis, 2010) or lack a rigorous stability analysis (Doya, 2000; Murray et al., 2002), except for Bhasin et al. (2012); Vamvoudakis et al. (2013).”
• Policy Iteration Adaptive Dynamic Programming Algorithm for Discrete-Time Nonlinear Systems (2014, background): “[46] proposed an online actor-critic-identifier architecture to obtain the optimal control law for uncertain nonlinear systems by policy iteration algorithms.”
• Reinforcement-Learning-Based Robust Controller Design for Continuous-Time Uncertain Nonlinear Systems Subject to Input Constraints (2015, methodology): “Traditionally, for utilizing RL approaches, a second NN called the action NN is introduced to approximate the control policy [23], [31]–[33], [39], [40].”
• Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning (2014, methodology): “Other than the IRL-based PI algorithms, efficient synchronous PI algorithms with guaranteed closed-loop stability were proposed in Bhasin et al. (2012), Modares, Naghibi-Sistani, and Lewis (2013), Vamvoudakis and Lewis (2010), to learn the solution to the HJB equation.”