Calibrating Deep Convolutional Gaussian Processes

By Gia-Lac Tran, Edwin V. Bonilla, John P. Cunningham, Pietro Michiardi, Maurizio Filippone. 2018

45 citations recorded by the source corpus.

The wide adoption of Convolutional Neural Networks (CNNs) in applications where decision-making under uncertainty is fundamental, has brought a great deal of attention to the ability of these models to accurately quantify the uncertainty in their predictions. Previous work on combining CNNs with Gaussian processes (GPs) has been developed under the assumption that the predictive probabilities of these models are well-calibrated. In this paper we show that, in fact, current combinations of CNNs and GPs are miscalibrated. We proposes a novel combination that considerably outperforms previous approaches on this aspect, while achieving state-of-the-art performance on image classification tasks.

Subfields: Artificial Intelligence, Control and Systems Engineering, Model Architectures, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Thomas Bo Schön
2. Fredrik Lindsten
3. Dustin Tran
4. Balaji Lakshminarayanan
5. Shreyas Padhy
6. Tania Bedrax-Weiss
7. Zi Lin
8. Dana Rezazadegan

In Control and Systems Engineering

1. Fredrik Lindsten
2. Abbas Khosravi
3. Mohammad Ghavamzadeh
4. Paul W. Fieguth
5. Saeid Nahavandi
6. Thomas Bo Schön
7. Balaji Lakshminarayanan
8. Dustin Tran

In Model Architectures

1. Abbas Khosravi
2. Mohammad Ghavamzadeh
3. Marc Niethammer
4. Xiaochun Cao
5. Paul W. Fieguth
6. Saeid Nahavandi
7. Thomas Bo Schön
8. Jeremiah Zhe Liu

In Probabilistic & Bayesian Methods

1. Balaji Lakshminarayanan
2. Dana Rezazadegan
3. Zi Lin
4. Vladimir Makarenkov
5. Abbas Khosravi
6. Shreyas Padhy
7. Mohammad Ghavamzadeh
8. Marc Niethammer

Citations in the corpus

• Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness (2020, methodology): “These approaches were shown to lead to poor calibration by recent work (Tran et al., 2019), which proposed a simple fix by combing MC Dropout with random Fourier features, which we term Calibrated Deep Gaussian Process (MCD-GP).”
• Local Temperature Scaling for Probability Calibration (2020, background): “6, 15 [63] Gia-Lac Tran, Edwin V Bonilla, John Cunningham, Pietro Michiardi, and Maurizio Filippone.”
• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges (2020, background): “than treating each component of weight matrix independently. As mentioned earlier, GPs were widely used for UQ in deep learning methods. Van der Wilk et al. [159], Blomqvist et al. [160], Tran et al. [161], Dutordoir et al. [162] and Shi et al. [163] introduced convolutional structure into GP. In an”
• Evaluating model calibration in classification (2019, methodology): “The first one encourages model calibration already during training (Lakshminarayanan, Pritzel, and Blundell 2017; Kumar, Sarawagi, and Jain 2018; Zhang et al. 2018; Kendall and Gal 2017; Tran et al. 2018, see, e.g., ).”