Reinforcement Learning for UAV Attitude Control

By William Koch, Renato Mancuso, R. West, Azer Bestavros. ACM Trans. Cyber Phys. Syst., 2018

473 citations recorded by the source corpus.

Autopilot systems are typically composed of an “inner loop” providing stability and control, whereas an “outer loop” is responsible for mission-level objectives, such as way-point navigation. Autopilot systems for unmanned aerial vehicles are predominately implemented using Proportional-Integral-Derivative (PID) control systems, which have demonstrated exceptional performance in stable environments. However, more sophisticated control is required to operate in unpredictable and harsh environments. Intelligent flight control systems is an active area of research addressing limitations of PID control most recently through the use of reinforcement learning (RL), which has had success in other applications, such as robotics. Yet previous work has focused primarily on using RL at the mission-level controller. In this work, we investigate the performance and accuracy of the inner control loop providing attitude control when using intelligent flight control systems trained with state-of-the-art RL algorithms—Deep Deterministic Policy Gradient, Trust Region Policy Optimization, and Proximal Policy Optimization. To investigate these unknowns, we first developed an open source high-fidelity simulation environment to train a flight controller attitude control of a quadrotor through RL. We then used our environment to compare their performance to that of a PID controller to identify if using RL is appropriate in high-precision, time-critical flight control.

Subfields: Aerospace Engineering, Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/b223b7d1dc76be5591bc261e9550ae4d168b6222

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

Philip S. Thomas

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

A Meta-MDP Approach to Exploration for Lifelong Reinforcement Learning (2019, background): “tions of PID controllers; where RL has gained signiﬁcant traction in recent years [30, 31]. One particular scenario were our proposed problem would arise is in using RL to control autonomous vehicles [9], where a single control policy would likely not work for a number of distinct vehicles and each ”
Dynamic and Multi-faceted Spatio-temporal Deep Learning for Traffic Speed Forecasting (2021, background): “The successful applications include but not limited to computer vision methods for traffic data collection [9] and regulations enforcement [2], reinforcement learning for automatic pilot [16, 18] and traffic signal control [6], big data analysis for human mobility modelling [10, 11], etc.”