Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof

By Asma Al-Tamimi, Frank L. Lewis, Murad Abu-Khalaf. IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics), 2008

1,009 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Biomedical Engineering, Computational Theory and Mathematics

View Paper: https://www.semanticscholar.org/paper/b60ac33923c255c504a02da8a2841a74eb79743c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Derong Liu
2. Ding Wang
3. Qinglai Wei
4. Huaguang Zhang
5. Frank L. Lewis
6. Yanhong Luo
7. Haibo He
8. Kyriakos G. Vamvoudakis

In Biomedical Engineering

1. Derong Liu
2. Qinglai Wei
3. Ding Wang
4. Frank L. Lewis
5. Huaguang Zhang
6. Yanhong Luo
7. Hanquan Lin
8. Kyriakos G. Vamvoudakis

In Computational Theory and Mathematics

1. Derong Liu
2. Qinglai Wei
3. Ding Wang
4. Frank L. Lewis
5. Huaguang Zhang
6. Haibo He
7. Yanhong Luo
8. Hamidreza Modares

Citations in the corpus

• Optimal Adaptive Control and Differential Games by Reinforcement Learning Principles (2012, methodology): “The convergence of value iteration using two neural networks for the discrete-time non-linear system (2.75) is proven in Al-Tamimi et al. (2008).”
• Adaptive Dynamic Programming for Control: A Survey and Recent Advances (2021, background): “Definition 1 (Admissible Control) [76]: A control u(t) is admissible with respect to the value function on a compact set ∈ Rn if u(t) is continuous on , u(0) = 0, u(t) stabilizes the systems on , and ∀x0 ∈ , V(x0) is finite.”
• Reinforcement Learning for Partially Observable Dynamic Processes: Adaptive Dynamic Programming Using Measured Output Data (2011, background): “In [2], it is shown that if a second approximator structure is assumed for the policy, then only the B matrix is needed for the policy improvement in Algorithm 1 or 3.”
• Value Iteration Adaptive Dynamic Programming for Optimal Control of Discrete-Time Nonlinear Systems (2016, methodology): “in [47], [52], [53], [55], and [59] can easily be justified by our developed value iteration algorithm.”
• An Equivalence Between Adaptive Dynamic Programming With a Critic and Backpropagation Through Time (2013, background): “Index Terms —Adaptive dynamic programming (ADP), backpropagation through time, dual heuristic programming (DHP), neural networks, value-gradient learning.”
• Decentralized Stabilization for a Class of Continuous-Time Nonlinear Interconnected Systems Using Online Learning Optimal Control Approach (2014, neutral): “Al-Tamimi et al. [18], Zhang et al. [23], and Liu et al. [26] 2162-237X © 2013 IEEE.”
• A Novel Infinite-Time Optimal Tracking Control Scheme for a Class of Discrete-Time Nonlinear Systems via the Greedy HDP Iteration Algorithm (2008, methodology): “Lemma 1 (cf., [ 2 ]): Let ˜ vi(k), k =0 , 1 ,...”
• A Clustering-Based Graph Laplacian Framework for Value Function Approximation in Reinforcement Learning (2014, methodology): “For dynamic systems without model information, Al-Tamimi and Lewis [19]”
• Policy Iteration Adaptive Dynamic Programming Algorithm for Discrete-Time Nonlinear Systems (2014, background): “For the optimal control problems, the developed control scheme must not only stabilize the control systems, but also make the performance index function finite, i.e., the admissible control law [39].”
• Adaptive Critic Nonlinear Robust Control: A Survey (2017, methodology): “ative ADP algorithm based on value iteration is important to the self-learning optimal control design of discrete-time systems [27], [47], [52], [55], [59], while the policy iteration is significant to the adaptive optimal control design of”
• Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming (2012, methodology): “Convergence analysis of the iterative algorithm The convergence analysis provided here is an extension of that given in Al-Tamimi et al. (2008).”
• Reinforcement Learning and Feedback Control: Using Natural Decision Methods to Design Optimal Adaptive Controllers (2012, methodology): “The convergence of value iteration using two neural networks for the discrete-time nonlinear system (50) is proven in [34].”
• Online adaptive optimal control for continuous-time Markov jump linear systems using a novel policy iteration algorithm (2015, neutral): “The extension to the nonlinear field was given by [29][28][15][27][26].”