Identifying Unknown Unknowns in the Open World: Representations and Policies for Guided Exploration

By Himabindu Lakkaraju, Ece Kamar, R. Caruana, Eric Horvitz. 2016

160 citations recorded by the source corpus.

Predictive models deployed in the real world may assign incorrect labels to instances with high confidence. Such errors or unknown unknowns are rooted in model incompleteness, and typically arise because of the mismatch between training data and the cases encountered at test time. As the models are blind to such errors, input from an oracle is needed to identify these failures. In this paper, we formulate and address the problem of informed discovery of unknown unknowns of any given predictive model where unknown unknowns occur due to systematic biases in the training data.We propose a model-agnostic methodology which uses feedback from an oracle to both identify unknown unknowns and to intelligently guide the discovery. We employ a two-phase approach which first organizes the data into multiple partitions based on the feature similarity of instances and the confidence scores assigned by the predictive model, and then utilizes an explore-exploit strategy for discovering unknown unknowns across these partitions. We demonstrate the efficacy of our framework by varying the underlying causes of unknown unknowns across various applications. To the best of our knowledge, this paper presents the first algorithmic approach to the problem of discovering unknown unknowns of predictive models.

Subfields: Artificial Intelligence, Reinforcement Learning, Anomaly & Outlier Detection

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Song-Chun Zhu
  2. Percy Liang
  3. Yoshua Bengio
  4. Mohit Bansal
  5. Sergey Levine
  6. Jianfeng Gao
  7. Quoc V. Le
  8. Luke S. Zettlemoyer

In Reinforcement Learning

  1. Enrique Munoz de Cote
  2. Josh Tenenbaum
  3. Jacob William Crandall
  4. Iyad Rahwan
  5. Pablo Hernandez-Leal
  6. Michael Kaisers
  7. Miroslav Dud'ik
  8. Dawn Song

In Anomaly & Outlier Detection

  1. Dawn Song
  2. Nicholas R. Jennings
  3. Hugo Larochelle

Citations in the corpus