Finite-Data Performance Guarantees for the Output-Feedback Control of an Unknown System

By Ross Boczar, Nikolai Matni, B. Recht. 2018

48 citations recorded by the source corpus.

As the systems we control become more complex, first-principle modeling becomes either impossible or intractable, motivating the use of machine learning techniques for the control of systems with continuous action spaces. As impressive as the empirical successes of these methods have been, strong theoretical guarantees of performance, safety, or robustness are few and far between. This paper takes a step towards providing such guarantees by establishing finite-data performance guarantees for the robust output-feedback control of an unknown FIR SISO system. In particular, we introduce the “Coarse-ID control” pipeline, which is composed of a system identification step followed by a robust controller synthesis procedure, and analyze its end-to-end performance, providing quantitative bounds on the performance degradation suffered due to model uncertainty as a function of the number of experiments used to identify the system. We conclude with numerical examples demonstrating the effectiveness of our method.

Subfields: Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Control and Systems Engineering

Citations in the corpus

Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guarantees (2018, methodology): “Boczar et al. (2018) provide ﬁnite-data guarantees for the “coarse-ID control” pipeline, which is composed of a system identiﬁcation step followed by a robust controller synthesis procedure.”
Robust Spectral Filtering and Anomaly Detection (2018, background): “Subsequently, a number of authors [11, 1, 14, 15, 3, 7] have applied them to the (Linear-Quadratic, LQ) control of an unknown system, which underlies much of reinforcement learning.”
Non-asymptotic Identification of LTI Systems from a Single Trajectory (2018, background): “On the other hand, the latter problem aims to learn a general purpose model that can be used in different control tasks, for instance, by combining it with robust control techniques [3, 4, 27].”
Data-Enabled Predictive Control: In the Shallows of the DeePC (2018, background): “In this spirit, an end-to-end ID and control pipeline is given in [25], [26] and arrives at a data-driven control solution with guarantees on the sample efﬁciency, stability, performance, and robustness.”
Tackling Climate Change with Machine Learning (2019, background): “Additionally, there has been recent progress in reinforcement learning for control [661–663], which could be useful for fine-tuning geoengineering interventions such as deciding where and when to release aerosols.”