Learning to fly by crashing

By Dhiraj Gandhi, Lerrel Pinto, Abhinav Kumar Gupta. 2017

299 citations recorded by the source corpus.

How do you learn to navigate an Unmanned Aerial Vehicle (UAV) and avoid obstacles? One approach is to use a small dataset collected by human experts: however, high capacity learning algorithms tend to overfit when trained with little data. An alternative is to use simulation. But the gap between simulation and real world remains large especially for perception problems. The reason most research avoids using large-scale real data is the fear of crashes! In this paper, we propose to bite the bullet and collect a dataset of crashes itself! We build a drone whose sole purpose is to crash into objects: it samples naive trajectories and crashes into random objects. We crash our drone 11,500 times to create one of the biggest UAV crash dataset. This dataset captures the different ways in which a UAV can crash. We use all this negative flying data in conjunction with positive data sampled from the same trajectories to learn a simple yet powerful policy for UAV navigation. We show that this simple self-supervised model is quite effective in navigating the UAV even in extremely cluttered environments with dynamic obstacles including humans. For supplementary video see:

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Control and Systems Engineering, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Reinforcement Learning

Citations in the corpus

Strike (With) a Pose: Neural Networks Are Easily Fooled by Strange Poses of Familiar Objects (2018, background): “For real-world technologies, such as self-driving cars [10], autonomous drones [14], and search-and-rescue robots [37], the test distribution may be non-stationary, and new observations will often be out-of-distribution (OoD), i.”
A 64-mW DNN-Based Visual Navigation Engine for Autonomous Nano-Drones (2018, background): “Recent results have shown that much lighter algorithms, based on convolutional neural networks (CNNs), are sufficient for enabling basic reactive navigation of small drones, even without a map of the environment [2], [17]–[20].”
BADGR: An Autonomous Self-Supervised Learning-Based Navigation System (2020, background): “Prior works have investigated learning navigation policies directly from real-world experience, but typically require a person [30, 33, 34] or SLAM algorithm [35] to gather the data, assume access to the ground-truth robot state [36], learn using low-bandwidth sensors [37], or only perform collision”
Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control (2018, background): “A number of recent works have studied self-supervised robotic learning, where large-scale unattended data collection is used to learn individual skills such as grasping [22], [23], [24], [25], pushgrasp synergies [26], or obstacle avoidance [27], [28].”
Learning to Drive in a Day (2018, background): “Variants of this approach have been demonstrated on self-driving cars [18], [19] and drones [20], [21], [22].”
Leave no Trace: Learning to Reset for Safe and Autonomous Reinforcement Learning (2017, background): “Even when tasks are designed so that these resets are easy (e.g. Levine et al. (2016) and Gu et al. (2017)), manual resets are necessary when the robot or environment breaks (e.g. Gandhi et al. (2017)).”
Continual Learning for Robotics (2019, background): “Nevertheless, a solution to this problem is to learn the policy in simulation and transfer it to deploy it in a real world robot (Bojarski et al., 2016; Rusu et al., 2016b; Gandhi et al., 2017; Kalifou et al., 2019).”
Self-Supervised Visual Planning with Temporal Skip Connections (2017, background): “Several prior works have focused on autonomous data collection for individual skills, such as grasping [7, 8] or obstacle avoidance [9, 10].”
Building Generalizable Agents with a Realistic and Rich 3D Environment (2018, background): “Dhiraj et al. (2017) collect a dataset of drones crashing into objects and train self-supervised agents on this data in order to avoid obstacles.”
Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout (2019, methodology): “For example, Levine et al. (2016) used an end-toend learned model to predict robotic actions from raw pixel data, Gupta et al.”
Robustness via Retrying: Closed-Loop Robotic Manipulation with Self-Supervised Learning (2018, background): “Many self-supervised robot learning methods have focused on predicting the outcome of a particular event, such as grasp success [1, 2, 3] or crashing [8, 9].”
Scaling data-driven robotics with reward sketching and batch reinforcement learning (2019, background): “Simple behaviours such as learning to grasp objects [35] or learning to fly [23] by avoiding crashing can”
Sim2Real Viewpoint Invariant Visual Servoing by Recurrent Control (2018, methodology): “Similarly to recent work on self-supervised robotic learning [18, 19, 2, 10, 26], our method uses observed images and self-supervision to train deep convolutional networks to predict action-conditioned task outcome.”
Learning Exploration Policies for Navigation (2019, methodology): “Learning based approaches (Gandhi et al., 2017; Sadeghi & Levine, 2017) have also been used to learn low-level collision avoidance policies.”
Audiovisual SlowFast Networks for Video Recognition (2020, background): “Beyond audio and visual modalities, extensive research has been conducted in other instantiations of multi-modal learning, including vision and language [20, 4, 83], vision and locomotion [87, 22], and learning from physiological data [48].”
An Introduction to Deep Reinforcement Learning (2018, unknown): “Deep RL also has potential for real-world applications such as robotics (Levine et al., 2016; Gandhi et al., 2017; Pinto et al., 2017), self-driving cars (You et al., 2017), finance (Deng et al., 2017) and smart grids (François-Lavet, 2017), to name a few.”
Embodied Question Answering in Photorealistic Environments With Point Cloud Perception (2019, background): “[19] collect a dataset of drone crashes and train self-supervised agents to avoid obstacles.”
GONet: A Semi-Supervised Deep Learning Approach For Traversability Estimation (2018, background): “But collecting negative examples, as in [21] or [22], can be time-consuming, costly, and dangerous.”
Combining Optimal Control and Learning for Visual Navigation in Novel Environments (2019, background): “Researchers have also employed learning to tackle locomotion problems [23, 24, 25, 26].”
How to train your robot with deep reinforcement learning: lessons we have learned (2021, background): “…to estimate uncertainty (Deisenroth and Rasmussen 2011; Eysenbach et al. 2018) of certain predictions, which can be a good indicator whether any unsafe behavior may happen, or can directly learn the probability of future unsafe behaviors from experience (Gandhi et al. 2017; Srinivasan et al. 2020).”