Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baseline

By Peng Wu, Xiaosong Jia, Li Chen, Junchi Yan, Hongyang Li, Yu Qiao. Neural Information Processing Systems, 2022

395 citations recorded by the source corpus.

Current end-to-end autonomous driving methods either run a controller based on a planned trajectory or perform control prediction directly, which have spanned two separately studied lines of research. Seeing their potential mutual benefits to each other, this paper takes the initiative to explore the combination of these two well-developed worlds. Specifically, our integrated approach has two branches for trajectory planning and direct control, respectively. The trajectory branch predicts the future trajectory, while the control branch involves a novel multi-step prediction scheme such that the relationship between current actions and future states can be reasoned. The two branches are connected so that the control branch receives corresponding guidance from the trajectory branch at each time step. The outputs from two branches are then fused to achieve complementary advantages. Our results are evaluated in the closed-loop urban driving setting with challenging scenarios using the CARLA simulator. Even with a monocular camera input, the proposed approach ranks first on the official CARLA Leaderboard, outperforming other complex candidates with multiple sensors or fusion mechanisms by a large margin. The source code is publicly available at https://github.com/OpenPerceptionX/TCP

Subfields: Automotive Engineering, Building and Construction, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/40717baa413da976571bf82410ea5542b0be86db

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Automotive Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Recent Advancements in End-to-End Autonomous Driving Using Deep Learning: A Survey (2023, methodology): “1) Behavioural Cloning: Behavioural cloning [7], [12], [13], [23], [34], [67], [78] is the supervised imitation learning task where the goal is to treat each state-action combination in expert distribution as an Independent and Identically Distributed (I.I.”
Planning-oriented Autonomous Driving (2022, background): “By contrast, the emergence of end-to-end autonomous driving [11,15,19,38,97] unites all nodes from perception, prediction and planning as a whole .”
NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking (2024, neutral): “For instance, an extensive body of end-to-end approaches focuses on closed-loop simulators, utilizing single-frame cameras, LiDAR point clouds, or a combination of both for expert imitation [7, 11, 36, 8, 52, 43, 44, 12, 24, 57, 22].”
End-to-End Autonomous Driving: Challenges and Frontiers (2023, methodology): “The policy distillation paradigm proposed in LBC [5] and related approaches [20], [21], [22], [23] has signiﬁcantly improved closed-loop performance by mimicking a well-behaved expert.”
DriveLM: Driving with Graph Visual Question Answering (2023, background): “Thus, it is possible to perform this functionality using a lightweight LLM [56] or driving-specific architecture that accepts a command as an input [25, 80].”
LMDrive: Closed-Loop End-to-End Driving with Large Language Models (2023, background): “TCP [51] proposed an ap-15121 proach that integrates the two branches for trajectory planning and direct control by involving a novel multi-step prediction.”
Parting with Misconceptions about Learning-based Vehicle Motion Planning (2023, methodology): “Ego-forecasting approaches include both end-to-end methods that utilize LiDAR scans [40, 41], RGB images [42, 43, 44, 45, 14, 46] or both [13, 47, 5], as well as modular methods involving lower-dimensional inputs like bird’s eye view (BEV) grids or state vectors [26, 48, 11, 49, 50].”