Control of a Quadrotor With Reinforcement Learning

By Jemin Hwangbo, Inkyu Sa, Roland Yves Siegwart, Marco Hutter. IEEE Robotics and Automation Letters, 2017

569 citations recorded by the source corpus.

In this letter, we present a method to control a quadrotor with a neural network trained using reinforcement learning techniques. With reinforcement learning, a common network can be trained to directly map state to actuator command making any predefined control structure obsolete for training. Moreover, we present a new learning algorithm that differs from the existing ones in certain aspects. Our algorithm is conservative but stable for complicated tasks. We found that it is more applicable to controlling a quadrotor than existing algorithms. We demonstrate the performance of the trained policy both in simulation and with a real quadrotor. Experiments show that our policy network can react to step response relatively accurately. With the same policy, we also demonstrate that we can stabilize the quadrotor in the air even under very harsh initialization (manually throwing it upside-down in the air with an initial velocity of 5 m/s). Computation time of evaluating the policy is only 7 $\mu$s per time step, which is two orders of magnitude less than common trajectory optimization algorithms with an approximated model.

Subfields: Aerospace Engineering, Cell Biology, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/9e09cff6cd4c0cfbe9675cd1f0ae67c9c75a8ac9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Cell Biology

In Control and Systems Engineering

Citations in the corpus

The current state and future outlook of rescue robotics (2019, background): “, 2016), or reinforcement learning (Andersson et al., 2015; Hwangbo et al., 2017) for control policies.”
weedNet: Dense Semantic Weed Classification Using Multispectral Images and MAV for Smart Farming (2017, background): “However, in the presence of large data, recent developments of deep learning have shown end-to-end learning approaches [22] outperforms traditional hand-crafted feature learning [6], [23].”
ToyBox: Better Atari Environments for Testing Reinforcement Learning Agents (2018, background): “Agents trained using deep RL can learn control policies for a variety of complex tasks using only sensory input [1, 9].”
Dynamic and Multi-faceted Spatio-temporal Deep Learning for Traffic Speed Forecasting (2021, background): “The successful applications include but not limited to computer vision methods for traffic data collection [9] and regulations enforcement [2], reinforcement learning for automatic pilot [16, 18] and traffic signal control [6], big data analysis for human mobility modelling [10, 11], etc.”
Reinforcement Learning for UAV Attitude Control (2018, methodology): “ rotor velocity to decrease the state space. Reward gains during training of TRPO and DDPG are quite inconsistent with large conﬁdence intervals. Learning performance is comparable to that observed in[8], where TRPO is able to accumulate more reward than DDPG during training for the task of navigati”
Transparency in Deep Reinforcement Learning Networks (2018, background): “Similarly, Fig.”
Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes (2024, background): “(64) developed the first DRL quadrotor control policy that was successfully validated on hardware for waypoint tracking and recovery from harsh initialization.”
Per-Contact Iteration Method for Solving Contact Dynamics (2018, methodology): “for reinforcement learning, which is implemented in RAI framework [16].”