End-to-End Urban Driving by Imitating a Reinforcement Learning Coach

By Zhejun Zhang, Alexander Liniger, Dengxin Dai, Fisher Yu, Luc Van Gool. IEEE International Conference on Computer Vision, 2021

337 citations recorded by the source corpus.

End-to-end approaches to autonomous driving commonly rely on expert demonstrations. Although humans are good drivers, they are not good coaches for end-to-end algorithms that demand dense on-policy supervision. On the contrary, automated experts that leverage privileged information can efficiently generate large scale on-policy and off-policy demonstrations. However, existing automated experts for urban driving make heavy use of hand-crafted rules and perform suboptimally even on driving simulators, where ground-truth information is available. To ad-dress these issues, we train a reinforcement learning expert that maps bird’s-eye view images to continuous low-level actions. While setting a new performance upper-bound on CARLA, our expert is also a better coach that provides in-formative supervision signals for imitation learning agents to learn from. Supervised by our reinforcement learning coach, a baseline end-to-end agent with monocular camera-input achieves expert-level performance. Our end-to-end agent achieves a 78% success rate while generalizing to a new town and new weather on the NoCrash-dense bench-mark and state-of-the-art performance on the more challenging CARLA LeaderBoard.

Subfields: Automotive Engineering, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/010ac9acd45c0073e79130e855aad3b65c51a5e1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Automotive Engineering

In Control and Systems Engineering

Citations in the corpus

ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learning (2022, background): “As we witness the blossom of end-to-end algorithms and success applications into various domains [38, 22], there are some attempt implementing such a philosophy in autonomous driving as well [41, 3, 15, 1, 12, 42, 53, 7, 8].”
VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planning (2024, background): “Existing learning-based planning methods [23, 19, 21, 40, 16, 54] follow a deterministic paradigm to directly regress the action.”
End-to-End Autonomous Driving: Challenges and Frontiers (2023, methodology): “The policy distillation paradigm proposed in LBC [5] and related approaches [20], [21], [22], [23] has signiﬁcantly improved closed-loop performance by mimicking a well-behaved expert.”
LMDrive: Closed-Loop End-to-End Driving with Large Language Models (2023, methodology): “Roach [54] employed a reinforcement learning agent with privileged access to environmental information and distilled a model as the final agent.”
Recent Advancements in End-to-End Autonomous Driving Using Deep Learning: A Survey (2023, methodology): “Another reinforcement learning expert, Roach [39], translates bird’s-eye view images into continuous low-level actions (see Fig.”
Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baseline (2022, methodology): “To help the agent better estimate its current state, we add a speed prediction head to predict current speed s from the image feature and a value prediction head to predict the expected return estimated by the expert, similarly as in [55].”
TransFuser: Imitation With Transformer-Based Sensor Fusion for Autonomous Driving (2022, neutral): “Building the expert with RL is also possible [111], [112] but it is more computationally demanding and less interpretable.”
Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer (2022, methodology): “With the aforementioned mimicking tricks, Roach [19] trained an RL-based privileged model as the expert agent to provide demonstrations for the IL agent.”
Planning-oriented Autonomous Driving (2022, background): “Pioneering works [14,16,21,22,78,95, 97,106] verified this vanilla design in the closed-loop simulation [26].”