Safely Learning to Control the Constrained Linear Quadratic Regulator

By Sarah Dean, Stephen Tu, Nikolai Matni, B. Recht. 2018

138 citations recorded by the source corpus.

We study the constrained linear quadratic regulator with unknown dynamics, addressing the tension between safety and exploration in data-driven control techniques. We present a framework which allows for system identification through persistent excitation, while maintaining safety by guaranteeing the satisfaction of state and input constraints. This framework involves a novel method for synthesizing robust constraint-satisfying feedback controllers, leveraging newly developed tools from system level synthesis. We connect statistical results with cost sub-optimality bounds to give non-asymptotic guarantees on both estimation and controller performance.

Subfields: Control and Systems Engineering, Reinforcement Learning, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Control and Systems Engineering

In Reinforcement Learning

In Safety, Robustness & Interpretability

Citations in the corpus

A Convergence Theory for Deep Learning via Over-Parameterization (2018, neutral): “Recent works in this line of research include [1, 8, 16, 17, 27–29, 39, 42, 48].”
Safe Reinforcement Learning Using Robust MPC (2019, background): “The only contribution providing robust constraint satisfaction guarantees is [22], where a linear feedback policy is learned.”
On the Sample Complexity of the Linear Quadratic Regulator (2017, methodology): “For example, in follow-up work we analyze robustness under the L 1 norm in the context of constraints on the states and control signals [14].”
On the model-based stochastic value gradient for continuous reinforcement learning (2020, background): “For ﬁnite-horizon non-discounted MDPs, the value V or action-conditional value Q of a policy π is which may be extended to regularize the policy’s entropy with some temperature α ≥ 0 as The value function of a given policy can be estimated ( i.e. policy evaluation ) by explicitly rolling out the wor”
Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challenges (2021, background): “In terms of hard constraints, the predominant approach is to take conservative actions to ensure that the hard constraints are satisﬁed at all times, despite the problem uncertainties [177].”
Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning (2021, background): “Ef-ﬁcient robot learning relies on multiple factors including control architecture de-sign (154), systematic training data collection (155), and appropriate function class selection (156).”