Posterior calibration and exploratory analysis for natural language processing models

By Khanh Nguyen, Brendan T. O'Connor. 2015

158 citations recorded by the source corpus.

Many models in natural language processing define probabilistic distributions over linguistic structures. We argue that (1) the quality of a model' s posterior distribution can and should be directly evaluated, as to whether probabilities correspond to empirical frequencies, and (2) NLP uncertainty can be projected not only to pipeline components, but also to exploratory data analysis, telling a user when to trust and not trust the NLP analysis. We present a method to analyze calibration, and apply it to compare the miscalibration of several commonly used models. We also contribute a coreference sampling algorithm that can create confidence intervals for a political event extraction task.

Subfields: Artificial Intelligence, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Natural Adversarial Examples (2019, methodology): “We estimate the RMS Calibration Error using adaptive binning [36, 22, 17].”
A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks (2016, neutral): “This is consistent with a great deal of anecdotal evidence from researchers (Nguyen & O’Connor, 2015; Yu et al., 2010; Provost et al., 1998; Nguyen et al., 2015).”
Unsolved Problems in ML Safety (2021, background): “…express their domain of competence in a more representative and meaningful way, researchers could further improve model calibration on typical testing data [68, 133, 113, 109, 205, 107, 108, 124], though the greater challenge is calibration on testing data that is unlike the training data [137].”
Deep Weighted Averaging Classifiers (2018, methodology): “In all cases we report accuracy and calibration (i.e., the accuracy of the predicted probabilities), measuring the latter in terms of mean absolute error (MAE), using the adaptive binning approach of Nguyen and O’Connor [38].”
Using Pre-Training Can Improve Model Robustness and Uncertainty (2019, background): “Nguyen & O’Connor (2015); Hendrycks & Gimpel (2017b) found that deep neural network classiﬁers display severe overcon-ﬁdence in their predictions, and that the problem becomes worse with increased representational capacity (Guo et al., 2017).”
Verified Uncertainty Calibration (2019, methodology): “The importance of this notion of calibration has been emphasized in personalized medicine [1], meteorological forecasting [2, 3, 4, 5, 6] and natural language processing applications [7, 8].”
Deep Anomaly Detection with Outlier Exposure (2018, methodology): “Prior research (Guo et al., 2017; Nguyen & O’Connor, 2015; Kuleshov & Liang, 2015) considers calibrating systems where test-time queries are samples from Din, but systems also encounter samples from Dtestout and should also ascribe low confidence to these samples.”
Learning-Based Single-Document Summarization with Compression and Anaphoricity Constraints (2016, background): “Fortunately, the coreference model’s posterior probabilities have been shown to be well-calibrated (Nguyen and O’Connor, 2015), meaning that cases where it is likely to make errors are signaled by ﬂatter posterior distributions.”
Accurate Uncertainties for Deep Learning Using Calibrated Regression (2018, background): “Calibrated conﬁdence estimates have been extensively studied by practitioners in medicine (Jiang et al., 2012), meteorology (Raftery et al., 2005), natural language processing (Nguyen and O’Connor, 2015), and other ﬁelds.”
Analyzing Uncertainty in Neural Machine Translation (2018, background): “…side against the right hand side where S is a set of 200 beam search hypotheses on the WMT’14 En-Fr validation set, covering an average of 22.4% of the model’s probability mass. Points are binned so that each point represents 10% of sentences in the validation or test set (Nguyen & O’Connor, 2015).”
Calibrating Deep Neural Networks using Focal Loss (2020, methodology): “We thus also consider another metric, AdaECE (Adaptive ECE), for which bin sizes are calculated so as to evenly distribute samples between bins (similar to the adaptive binning procedure in [25]): AdaECE = ∑M i=1 |Bi| N |Ai − Ci| s.”
Recalibrating classifiers for interpretable abusive content detection (2020, background): “This has received considerable attention in machine learning and NLP research (Guo et al., 2017; Pleiss et al., 2017; Nguyen and O’Connor, 2015).”
CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing (2023, background): “[125] Khanh Nguyen and Brendan O’Connor.”
Calibration of Pre-trained Transformers (2020, background): “However, past work has not analyzed large-scale, pre-trained models, and we additionally analyze out-of-domain settings, whereas past work largely focuses on in-domain calibration (Nguyen and O’Connor, 2015; Guo et al., 2017).”
Teaching Models to Express Their Uncertainty in Words (2022, background): “The problem of truthfulness motivates calibration for language models (Nguyen & O’Connor, 2015).”
Text and Causal Inference: A Review of Using Text to Remove Confounding from Causal Estimates (2020, background): “…with text classifiers, but there are still open problems in accounting for error arising from other text representations and issues of calibration (Nguyen and O’Connor, 2015) and prevalence estimation (Card and Smith, 2018; Keith and O’Connor, 2018) in conjunction with NLP. Ideas from the large…”