Bayes-Probe: Distribution-Guided Sampling for Prediction Level Sets
By Serena Booth, Yilun Zhou, Ankit J. Shah, Julie Arnold Shah. 2020
Building machine learning models requires a suite of tools for interpretation, understanding, and debugging. Many existing methods have been proposed, but it can still be difficult to probe for examples which communicate model behaviour. We introduce Bayes-Probe, a model inspection method for analyzing neural networks by generating distribution-conforming examples of known prediction confidence. By selecting appropriate distributions and confidence prediction values, Bayes-Probe can be used to synthesize ambivalent predictions, uncover in-distribution adversarial examples, and understand novel-class extrapolation and domain adaptation behaviours. Bayes-Probe is model agnostic, requiring only a data generator and classifier prediction. We use Bayes-Probe to analyze models trained on both procedurally-generated data (CLEVR) and organic data (MNIST and Fashion-MNIST). Code is available at https://github.com/serenabooth/Bayes-Probe.
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
- Javier Antorán
- Umang Bhatt
- Adrian Weller
- Tameem Adel
- José Miguel Hernández-Lobato
- Yarin Gal
- Yingzhen Li
- Richard E. Turner
Citations in the corpus
- Getting a CLUE: A Method for Explaining Uncertainty Estimates (2020, methodology): “Generative-model based methods methods are counterfactual because they do not assign importance to the observed features but rather propose alternative features based on the data manifold Chang et al. (2019). This is the case for FIDO and CLUE.”