Variational Autoencoder for End-to-End Control of Autonomous Driving with Novelty Detection and Training De-biasing

By Alexander Amini, Wilko Schwarting, Guy Rosman, Brandon Araki, Sertac Karaman, Daniela Rus. 2018

83 citations recorded by the source corpus.

This paper introduces a new method for end-to-end training of deep neural networks (DNNs) and evaluates it in the context of autonomous driving. DNN training has been shown to result in high accuracy for perception to action learning given sufficient training data. However, the trained models may fail without warning in situations with insufficient or biased training data. In this paper, we propose and evaluate a novel architecture for self-supervised learning of latent variables to detect the insufficiently trained situations. Our method also addresses training data imbalance, by learning a set of underlying latent variables that characterize the training data and evaluate potential biases. We show how these latent distributions can be leveraged to adapt and accelerate the training pipeline by training on only a fraction of the total dataset. We evaluate our approach on a challenging dataset for driving. The data is collected from a full-scale autonomous vehicle. Our method provides qualitative explanation for the latent variables learned in the model. Finally, we show how our model can be additionally trained as an end-to-end controller, directly outputting a steering control command for an autonomous vehicle.

Subfields: Artificial Intelligence, Automotive Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Automotive Engineering

Citations in the corpus

Uncovering and Mitigating Algorithmic Bias through Learned Latent Structure (2019, methodology): “Our algorithm is applicable to a wide range of computer vision tasks and has also been used to successfully debias end-to-end autonomous vehicle controllers [3].”
Exploring the Limitations of Behavior Cloning for Autonomous Driving (2019, background): “Existing mitigation strategies currently need more informative intermediate representations, either learned [3] or using strong domain knowledge [5].”
Variational End-to-End Navigation and Localization (2018, background): “A recent line of work has tied end-to-end driving networks to variational inference [19], allowing us to handle cases where multiple actions are possible, as well as reason about robustness, atypical data, and dataset normalization.”
AUTO-ENCODING VARIATIONAL BAYES (2020, unknown): “Researchers have attempted to use VAEs for decision-making applications too, including novelty detection in control applications [4], mutation-effect prediction from genomic sequences [5], artifact detection [6], and Bayesian hypothesis testing for single-cell RNA sequencing data [7, 8].”