Learning-Based Model Predictive Control for Safe Exploration

By Torsten Koller, Felix Berkenkamp, Matteo Turchetta, Andreas Krause. IEEE Conference on Decision and Control, 2018

439 citations recorded by the source corpus.

Learning-based methods have been successful in solving complex control tasks without significant prior knowledge about the system. However, these methods typically do not provide any safety guarantees, which prevents their use in safety-critical, real-world applications. In this paper, we present a learning-based model predictive control scheme that can provide provable high-probability safety guarantees. To this end, we exploit regularity assumptions on the dynamics in terms of a Gaussian process prior to construct provably accurate confidence intervals on predicted trajectories. Unlike previous approaches, we do not assume that model uncertainties are independent. Based on these predictions, we guarantee that trajectories satisfy safety constraints. Moreover, we use a terminal set constraint to recursively guarantee the existence of safe control actions at every iteration. In our experiments, we show that the resulting algorithm can be used to safely and efficiently explore and learn about dynamic systems.

Subfields: Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/ede9f86e18d39993589963205d56de7a926c7196

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Control and Systems Engineering

Citations in the corpus

Cautious Model Predictive Control Using Gaussian Process Regression (2017, background): “While under certain assumptions on g, some strict overapproximations of resulting confidence sets exist [20], and they are typically conservative and computationally demanding.”
A Review of Safe Reinforcement Learning: Methods, Theories, and Applications (2024, positive): “Furthermore, they reviewed a large number of papers that are related to control theory and analyzed how to use control theory to guarantee RL safety and stability, such as using model predictive control[87],adaptivecontrol[88],robustcontrol[89],Lyapunov functions[35]forRLstabilityandsafety.”
End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks (2019, methodology): “…methods or model predictive control to guarantee safety under system dynamics during learning (Wang, Theodorou, and Egerstedt 2017; Berkenkamp et al. 2017; Chow et al. 2018; Ohnishi et al. 2018; Koller et al. 2018), but they do not address the issue of exploration and performance optimization.”
A Review of Safe Reinforcement Learning: Methods, Theory and Applications (2022, background): “control [17, 132, 139, 183, 204, 231, 258], adaptive control [89, 115, 199, 238], robust control [26, 73, 112, 282], lyapunov function [56, 57] for RL stability and safety.”
On the model-based stochastic value gradient for continuous reinforcement learning (2020, background): “For ﬁnite-horizon non-discounted MDPs, the value V or action-conditional value Q of a policy π is which may be extended to regularize the policy’s entropy with some temperature α ≥ 0 as The value function of a given policy can be estimated ( i.e. policy evaluation ) by explicitly rolling out the wor”
Learning-Based Model Predictive Control: Toward Safe Learning in Control (2020, methodology): “A data-based optimization technique with several applications to MPC that has received these results in this review but refer readers to References 13–15 for a more detailed discussion of the approach and resulting MPC schemes.”