Agile Autonomous Driving using End-to-End Deep Imitation Learning

By Yunpeng Pan, Ching-An Cheng, Kamil Saigol, Keuntaek Lee, Xinyan Yan, Evangelos A. Theodorou, Byron Boots. 2017

285 citations recorded by the source corpus.

We present an end-to-end imitation learning system for agile, off-road autonomous driving using only low-cost sensors. By imitating a model predictive controller equipped with advanced sensors, we train a deep neural network control policy to map raw, high-dimensional observations to continuous steering and throttle commands. Compared with recent approaches to similar tasks, our method requires neither state estimation nor on-the-fly planning to navigate the vehicle. Our approach relies on, and experimentally validates, recent imitation learning theory. Empirically, we show that policies trained with online imitation learning overcome well-known challenges related to covariate shift and generalize better than policies trained with batch imitation learning. Built on these insights, our autonomous driving system demonstrates successful high-speed off-road driving, matching the state-of-the-art performance.

Subfields: Artificial Intelligence, Automotive Engineering, Computer Vision and Pattern Recognition, Reinforcement Learning, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Automotive Engineering

In Computer Vision and Pattern Recognition

In Reinforcement Learning

In Robotics & Control

Citations in the corpus

Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems (2020, background): “Imitation learning has been a popular approach towards end-toend, data-driven methods in autonomous driving (Bojarski et al., 2016; Sun et al., 2018a; Pan et al., 2017; Codevilla et al., 2018).”
Ensemble Bayesian Decision Making with Redundant Deep Perceptual Control Policies (2018, background): “this task, difficulty arises in mapping sensor inputs into driving commands [4].”
Deep Drone Racing: Learning Agile Flight in Dynamic Environments (2018, background): “While being independent of any global map and position estimate, these methods are not directly applicable to our specific problem due to their high computational complexity [27, 11], their low maximum speed [13] or the inherent difficulties of generalizing to 3D motions [25, 4, 24].”
Convergence of Value Aggregation for Imitation Learning (2018, background): “In this case, it can be shown that CR(x) ≥ F (x, x) for some C > 0 (usually C > 1) (Pan et al., 2017).”
Fighting Failures with FIRE: Failure Identification to Reduce Expert Burden in Intervention-Based Learning (2020, background): “This solution, and other related work, is very effective when a programmatic expert is available to autonomously relabel states [7], but for many robotics tasks, the only expert available is a human being.”
Recent Advances in Robot Learning from Demonstration (2020, methodology): “Since this early success, LfD-based autonomous navigation for ground vehicles has been attempted using IRL (117, 121, 168), interactive learning (169), active learning (170), adversarial learning (171), and end-to-end learning (40, 172).”
Provable Representation Learning for Imitation Learning via Bi-level Optimization (2020, background): “Imitation learning can help with sample efficiency of many problems (Ross & Bagnell, 2010; Sun et al., 2017; Daumé et al., 2009; Chang et al., 2015; Pan et al., 2018).”
Early Failure Detection of Deep End-to-End Control Policy by Reinforcement Learning (2019, methodology): “Our earlier work, based on DAgger, specifically addressed the problem of learning a MPC policy for high-speed autonomous navigation [3], but did not account for safety and robustness in the learned model.”
A Survey of Deep Learning Applications to Autonomous Vehicle Control (2019, methodology): “[126] used DAgger-like imitation learning to learn to drive at high speeds autonomously, with continuous actions for both steering and acceleration.”
Conditional Vehicle Trajectories Prediction in CARLA Urban Environment (2019, methodology): “[23] use a Model Predictive Control agent with high end sensors to train for aggressive offroad driving.”
End-to-End Autonomous Driving: Challenges and Frontiers (2023, methodology): “,severalpapers[10],[24],[25]have proposedaggregatingon-policydata[26]duringtraining.”
Learning by Cheating (2019, methodology): “This enables automatic DAgger-like training in which supervision from the privileged agent is gathered adaptively via online rollouts of the sensorimotor agent [16, 19, 23].”
A Survey of End-to-End Driving: Architectures and Training Methods (2020, methodology): “When using an algorithm as the expert that provides the true labels [11], [51], one can do imitation learning on-policy – the learning agent controls the car during data collection while continuously getting supervision from the expert algorithm.”
End-to-End Urban Driving by Imitating a Reinforcement Learning Coach (2021, background): “[34] demonstrated endto-end off-road racing with a monocular camera by imitating a model predictive control expert with access to expensive sensors.”
Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimism (2021, background): “This paradigm is useful in a diverse array of application domains such as healthcare [51, 15, 35], autonomous driving [57, 4, 36], and recommendation systems [47, 12, 49].”
DREAM Architecture: a Developmental Approach to Open-Ended Learning in Robotics (2020, unknown): “Deep learning is the only approach so far that has been able to deal with end-to-end control [5, 19, 48].”
Deep Local Trajectory Replanning and Control for Robot Navigation (2019, background): “Even though this is computationally more expensive, prior work suggests that it can improve imitation [57, 58].”