A Survey of End-to-End Driving: Architectures and Training Methods

By Ardi Tampuu, Tambet Matiisen, Maksym Semikin, Dmytro Fishman, Naveed Muhammad. IEEE Transactions on Neural Networks and Learning Systems, 2020

323 citations recorded by the source corpus.

Autonomous driving is of great interest to industry and academia alike. The use of machine learning approaches for autonomous driving has long been studied, but mostly in the context of perception. In this article, we take a deeper look on the so-called end-to-end approaches for autonomous driving, where the entire driving pipeline is replaced with a single neural network. We review the learning methods, input and output modalities, network architectures, and evaluation schemes in end-to-end driving literature. Interpretability and safety are discussed separately, as they remain challenging for this approach. Beyond providing a comprehensive overview of existing methods, we conclude the review with an architecture that combines the most promising elements of the end-to-end autonomous driving systems.

Subfields: Artificial Intelligence, Automotive Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/2dbc968d0e2fe7592473f2c4857c5e8b8b6c58c7

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Automotive Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Milestones in Autonomous Driving and Intelligent Vehicles: Survey of Surveys (2023, unknown): “prediction [73], [74], planning [75], [76], [77], [78], [79], [80], E2E [81], [82], control [83], [84], [85], [86], [87], [88], [89], system [90], [91], [92], [93], hardware [94], [95], [96], soft-”
Motion Planning for Autonomous Driving: The State of the Art and Future Perspectives (2023, background): “The end-to-end approach has emerged as a popular research direction in recent years, as demonstrated by previous work [1], [9], which illustrates methods for mapping raw perception inputs to control command outputs.”
DriveLM: Driving with Graph Visual Question Answering (2023, background): “The inadequacy of generalization to the “long tail” of corner cases poses significant safety concerns to AD systems [8, 65, 66].”
Driving Into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving (2023, background): “The emergence of end-to-end autonomous driving [29, 30, 54] has recently garnered increasing attention.”
Recent Advancements in End-to-End Autonomous Driving Using Deep Learning: A Survey (2023, background): “A few end-to-end surveys, including the work by Tampuu et al. [28], provide an architectural overview of the complete end-to-end driving pipeline.”
Computing Systems for Autonomous Driving: State of the Art and Challenges (2020, background): “2) Cityscapes: For the semantic understanding of urban street scenes, the Cityscapes [57] dataset includes 2-D semantic segmentation on pixel-level, instance-level, and panoptic semantic labeling, and provides corresponding benchmarks on them.”
End-to-End Autonomous Driving: Challenges and Frontiers (2023, background): “We would like to clarify the difference between our survey and previous related surveys [40], [41], [42], [43], [44], [45], [46], [47], [48].”