Safe Reinforcement Learning Using Robust MPC

By Mario Zanon, Sebastien Gros. IEEE Transactions on Automatic Control, 2019

342 citations recorded by the source corpus.

Reinforcement learning (RL) has recently impressed the world with stunning results in various applications. While the potential of RL is now well established, many critical aspects still need to be tackled, including safety and stability issues. These issues, while secondary for the RL community, are central to the control community that has been widely investigating them. Model predictive control (MPC) is one of the most successful control techniques because, among others, of its ability to provide such guarantees even for uncertain constrained systems. Since MPC is an optimization-based technique, optimality has also often been claimed. Unfortunately, the performance of MPC is highly dependent on the accuracy of the model used for predictions. In this article, we propose to combine RL and MPC in order to exploit the advantages of both, and therefore, obtain a controller that is optimal and safe. We illustrate the results with two numerical examples in simulations.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/5020d1dce1392dd6c5621ac55fa573b3aa541eaa

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yaodong Yang
2. Jun Wang
3. Shangding Gu
4. Yali Du
5. Alois C. Knoll
6. Guang Chen
7. Frank Allgöwer
8. Florian Walter

In Computational Theory and Mathematics

1. Shangding Gu
2. Yali Du
3. Alois C. Knoll
4. Guang Chen
5. Frank Allgöwer
6. Florian Walter
7. Jun Wang
8. Matthias Albrecht Müller

In Control and Systems Engineering

1. Shangding Gu
2. Yali Du
3. Alois C. Knoll
4. Guang Chen
5. Frank Allgöwer
6. Florian Walter
7. Longyu Yang
8. Jun Wang

Citations in the corpus

• Data-Driven Model Predictive Control With Stability and Robustness Guarantees (2019, methodology): “In various recent works, learning-based or adaptive MPC schemes have been proposed, which improve an inaccurate initial model using online measurements [5]–[9], while giving guarantees on the resulting closed loop.”
• A Review of Safe Reinforcement Learning: Methods, Theory and Applications (2022, methodology): “Moreover, some safe RL methods are proposed from the perspective of Model Predictive Control (MPC) [321], e.”
• A Review of Safe Reinforcement Learning: Methods, Theories, and Applications (2024, positive): “In addition, some safe RL methods are proposed from the perspectiveofModelPredictiveControl(MPC)[123],e.g.,MPC isusedtomakerobustdecisionsinCMDPs[124]byleveraginga constrainedMPCmethod[125],whichalsointroducesageneral safetyframeworktomakedecisions[126].”