Safe controller optimization for quadrotors with Gaussian processes

By Felix Berkenkamp, Angela P. Schoellig, Andreas Krause. 2015

335 citations recorded by the source corpus.

One of the most fundamental problems when designing controllers for dynamic systems is the tuning of the controller parameters. Typically, a model of the system is used to obtain an initial controller, but ultimately the controller parameters must be tuned manually on the real system to achieve the best performance. To avoid this manual tuning step, methods from machine learning, such as Bayesian optimization, have been used. However, as these methods evaluate different controller parameters on the real system, safety-critical system failures may happen. In this paper, we overcome this problem by applying, for the first time, a recently developed safe optimization algorithm, SafeOpt, to the problem of automatic controller parameter tuning. Given an initial, low-performance controller, SafeOpt automatically optimizes the parameters of a control law while guaranteeing safety. It models the underlying performance measure as a Gaussian process and only explores new controller parameters whose performance lies above a safe performance threshold with high probability. Experimental results on a quadrotor vehicle indicate that the proposed method enables fast, automatic, and safe optimization of controller parameters without human intervention.

Subfields: Artificial Intelligence, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

Citations in the corpus

Safe Model-based Reinforcement Learning with Stability Guarantees (2017, methodology): “These methods are used in [19] to safely optimize a parametric control policy on a quadrotor.”
Bayesian optimization with safety constraints: safe and automatic parameter tuning in robotics (2016, positive): “n this section, we review Gaussian processes (GPs) and Bayesian Optimization, which form the foundation of our safe Bayesian optimization algorithm. The introduction to GPs is standard and taken from [2] and [11]. A. Gaussian Process (GP) Both the function f(a) and the safety constraints g i(a) in S”
A survey of uncertainty in deep neural networks (2021, background): “reinforcement learning algorithms are backed up by stability guarantees for safe interactions in the real-world [312], [313], [314].”
Learning-Based Model Predictive Control: Toward Safe Learning in Control (2020, methodology): “While this scheme can be applied to most parameterized controllers (102, 103), only a few techniques explicitly consider the parameterization of optimal control formulations.”
Safe Exploration in Finite Markov Decision Processes with Gaussian Processes (2016, background): “To achieve this, the function is cautiously explored, starting from a set of points that is known to be safe initially.”
Neural Lander: Stable Drone Landing Control Using Learned Dynamics (2018, methodology): “[4] employs Bayesian Optimization for open-air control but not for takeoff/landing.”