Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles

By Balaji Lakshminarayanan, Alexander Pritzel, Charles Blundell. 2016

8,039 citations recorded by the source corpus.

Deep neural networks (NNs) are powerful black box predictors that have recently achieved impressive performance on a wide spectrum of tasks. Quantifying predictive uncertainty in NNs is a challenging and yet unsolved problem. Bayesian NNs, which learn a distribution over weights, are currently the state-of-the-art for estimating predictive uncertainty; however these require significant modifications to the training procedure and are computationally expensive compared to standard (non-Bayesian) NNs. We propose an alternative to Bayesian NNs that is simple to implement, readily parallelizable, requires very little hyperparameter tuning, and yields high quality predictive uncertainty estimates. Through a series of experiments on classification and regression benchmarks, we demonstrate that our method produces well-calibrated uncertainty estimates which are as good or better than approximate Bayesian NNs. To assess robustness to dataset shift, we evaluate the predictive uncertainty on test examples from known and unknown distributions, and show that our method is able to express higher uncertainty on out-of-distribution examples. We demonstrate the scalability of our method by evaluating predictive uncertainty estimates on ImageNet.

Subfields: Artificial Intelligence, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Sergey Levine
  2. Yoshua Bengio
  3. Mohit Bansal
  4. Percy Liang
  5. Quoc V. Le
  6. Jianfeng Gao
  7. Ruslan Salakhutdinov
  8. Luke S. Zettlemoyer

In Evaluation & Benchmarks

  1. Yejin Choi
  2. Mohit Bansal
  3. Percy Liang
  4. Hannaneh Hajishirzi
  5. Yoshua Bengio
  6. Sergey Levine
  7. Robin Jia
  8. Noah A. Smith

Citations in the corpus