Introspective perception: Learning to predict failures in vision systems

By Shreyansh Daftry, Sam Zeng, J. Bagnell, Martial Hebert. 2016

86 citations recorded by the source corpus.

As robots aspire for long-term autonomous operations in complex dynamic environments, the ability to reliably take mission-critical decisions in ambiguous situations becomes critical. This motivates the need to build systems that have situational awareness to assess how qualified they are at that moment to make a decision. We call this self-evaluating capability as introspection. In this paper, we take a small step in this direction and propose a generic framework for introspective behavior in perception systems. Our goal is to learn a model to reliably predict failures in a given system, with respect to a task, directly from input sensor data. We present this in the context of vision-based autonomous MAV flight in outdoor natural environments, and show that it effectively handles uncertain situations.

Subfields: Artificial Intelligence, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

Citations in the corpus

Uncertainty-Aware Reinforcement Learning for Collision Avoidance (2017, background): “While it is possible to train a neural network model that outputs a mean and a variance as its prediction [2], this model is not in general guaranteed to output high variances for unfamiliar inputs because the network is by deﬁnition trained only on the datapoints that are in the training set.”
A survey of uncertainty in deep neural networks (2021, background): “Other approaches also learn on the robots’ past experiences of failures or detect inconsistencies of the predictors [332], [333].”