Provably safe and robust learning-based model predictive control

By Anil Jayanti Aswani, Humberto González, S. Sastry, Claire Jennifer Tomlin. 2011

575 citations recorded by the source corpus.

Controller design faces a trade-off between robustness and performance, and the reliability of linear controllers has caused many practitioners to focus on the former. However, there is renewed interest in improving system performance to deal with growing energy constraints. This paper describes a learning-based model predictive control (LBMPC) scheme that provides deterministic guarantees on robustness, while statistical identification tools are used to identify richer models of the system in order to improve performance; the benefits of this framework are that it handles state and input constraints, optimizes system performance with respect to a cost function, and can be designed to use a wide variety of parametric or nonparametric statistical tools. The main insight of LBMPC is that safety and performance can be decoupled under reasonable conditions in an optimization framework by maintaining two models of the system. The first is an approximate model with bounds on its uncertainty, and the second model is updated by statistical methods. LBMPC improves performance by choosing inputs that minimize a cost subject to the learned dynamics, and it ensures safety and robustness by checking whether these same inputs keep the approximate model stable when it is subject to uncertainty. Furthermore, we show that if the system is sufficiently excited, then the LBMPC control action probabilistically converges to that of an MPC computed using the true dynamics.

Subfields: Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Control and Systems Engineering

Citations in the corpus

Verifiable Reinforcement Learning via Policy Extraction (2018, background): “Veriﬁcation of learned controllers [24, 32, 3, 20, 19, 31] is a crucial component of many such systems [2, 8], but existing approaches do not scale to high dimensional state spaces.”
Safe Model-based Reinforcement Learning with Stability Guarantees (2017, background): “In this setting, robust feasability and constraint satisfaction can be guaranteed for a learned model with bounded errors using robust model predictive control [25].”
Bootstrapping bilinear models of Simple Vehicles (2015, background): “The environment is a probability distribution on Maps that generates the map seen at each episode.”
Safe Reinforcement Learning Using Robust MPC (2019, background): “Thecombination of learning and control techniques has been proposed in, e.g., [13], [14], [15], [16], [17].”
Safe Control under Uncertainty (2015, background): “More recently, safe learning approaches construct controllers that keep the system in the safe region, while the optimal strategy is learned online [15, 3, 1].”
Learning-Based Model Predictive Control for Safe Exploration (2018, background): “Theoretical safety guarantees in learning-based MPC (LBMPC) are established in [6].”
A Review of Safe Reinforcement Learning: Methods, Theory and Applications (2022, methodology): “control [17, 132, 139, 183, 204, 231, 258], adaptive control [89, 115, 199, 238], robust control [26, 73, 112, 282], lyapunov function [56, 57] for RL stability and safety.”
Safely Learning to Control the Constrained Linear Quadratic Regulator (2018, methodology): “Strategies for incorporating estimation of the dynamics include experiment-design inspired costs [12], decoupling learning from constraint satisfaction [13], and set-membership methods rather than parameter estimation [14].”
Learning to Synthesize Programs as Interpretable and Generalizable Policies (2021, background): “[77] Anil Aswani, Humberto Gonzalez, S Shankar Sastry, and Claire Tomlin.”
Hamilton-Jacobi reachability: A brief overview and recent advances (2017, background): “Concrete practical applications include aircraft auto-landing [5], automated aerial refueling [6], model predictive control (MPC) of quadrotors [7], [8], multiplayer reach-avoid games [9], largescale multiple-vehicle path planning [10], [11], and realtime safe motion planning [12].”
Guaranteed Safe Online Learning via Reachability: tracking a ground target using a quadrotor (2012, background): “(In particular, Learning-Based Model Predictive Control requires a linear model of the system dynamics for its safety guarantees to hold, which is not necessary under GSOLR [11].)”
Safe controller optimization for quadrotors with Gaussian processes (2015, background): “Stability can be guaranteed by ensuring that either the controller is robustly stable for all possible models within the uncertainty specification [15] or the system never leaves a safe subset of the state space [16], [17], [18].”
A Review of Safe Reinforcement Learning: Methods, Theories, and Applications (2024, unknown): “Inthis section,wewillinvestigatemodel-basedsafeRLmethodsfrom differentperspectives:policyoptimization-basedapproaches, controltheory-basedapproaches,formalmethods-basedap-proaches, and Gaussian process-based approaches.”
A General Safety Framework for Learning-Based Control in Uncertain Robotic Systems (2017, methodology): “The robust model-predictive control approach in [17] learns”