Early Failure Detection of Deep End-to-End Control Policy by Reinforcement Learning

By Keuntaek Lee, Kamil Saigol, Evangelos A. Theodorou. 2019

9 citations recorded by the source corpus.

We propose the use of Bayesian networks, which provide both a mean value and an uncertainty estimate as output, to enhance the safety of learned control policies under circumstances in which a test-time input differs significantly from the training set. Our algorithm combines reinforcement learning and end-to-end imitation learning to simultaneously learn a control policy as well as a threshold over the predictive uncertainty of the learned model, with no hand-tuning required. Corrective action, such as a return of control to the model predictive controller or human expert, is taken before the failure of tasks, when the uncertainty threshold is exceeded. We validate our method on fully-observable and vision-based partially-observable systems using cart-pole and autonomous driving simulations using deep convolutional Bayesian neural networks. We demonstrate that our method is robust to uncertainty resulting from varying system dynamics as well as from partial state observability.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Ensemble Bayesian Decision Making with Redundant Deep Perceptual Control Policies (2018, methodology): “It is using the VGG 16 [19]-like network, with modifications to include dropout at each layer as well as output the variance, σ̂ stemming from aleatoric uncertainty [20].”