Variational End-to-End Navigation and Localization

By Alexander Amini, Guy Rosman, Sertac Karaman, Daniela Rus. 2018

124 citations recorded by the source corpus.

Deep learning has revolutionized the ability to learn “end-to-end” autonomous vehicle control directly from raw sensory data. While there have been recent extensions to handle forms of navigation instruction, these works are unable to capture the full distribution of possible actions that could be taken and to reason about localization of the robot within the environment. In this paper, we extend end-to-end driving networks with the ability to perform point-to-point navigation as well as probabilistic localization using only noisy GPS data. We define a novel variational network capable of learning from raw camera data of the environment as well as higher level roadmaps to predict (1) a full probability distribution over the possible control commands; and (2) a deterministic control command capable of navigating on the route specified within the map. Additionally, we formulate how our model can be used to localize the robot according to correspondences between the map and the observed visual road topology, inspired by the rough localization that human drivers can perform. We test our algorithms on real-world driving data that the vehicle has never driven through before, and integrate our point-topoint navigation algorithms onboard a full-scale autonomous vehicle for real-time performance. Our localization algorithm is also evaluated over a new set of roads and intersections to demonstrates rough pose localization even in situations without any GPS prior.

Subfields: Automotive Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Automotive Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Uncovering and Mitigating Algorithmic Bias through Learned Latent Structure (2019, background): “For example, ML and artificial intelligence (AI) are already being used to control end-to-end autonomous vehicles [2], how long a criminal should spend in prison [6], the order in which a person is presented the news [27], or even diagnose and treat medical patients [24].”
Learning to Walk in the Real World with Minimal Human Effort (2020, background): “robotic applications, including autonomous driving [5, 41], navigation [15], and manipulation [33, 36, 2].”
Combining Optimal Control and Learning for Visual Navigation in Novel Environments (2019, background): “Several works also use end-to-end learning for navigation using laser scans [12, 13, 14], for training and combining a local planner with a higher level roadmap for long range navigation [15, 16, 17, 18], or visual servoing [19].”
Deep Evidential Regression (2019, methodology): “This method is widely applicable across regression tasks including temporal forecasting [17], property prediction [8], and control learning [1, 30].”
A Survey of End-to-End Driving: Architectures and Training Methods (2020, background): “some end-to-end models are only open-loop evaluated [8], [10], [69], [85], [86], [89], [114].”