Calibration of Confidence Measures in Speech Recognition

By Dong Yu, Jinyu Li, Li Deng. 2011

82 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Deep Learning: Methods and Applications (2014, methodology): “For example, the deep-structured CRF, which stacks many layers of CRFs, have been usefully applied to the task of language identification [429], phone recognition [410], sequential labeling in natural language processing [428], and confidence calibration in speech recognition [423].”
Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces (2018, background): “Confidence scoring is a notoriously hard problem in speech recognition [20, 59].”
A tutorial survey of architectures, algorithms, and applications for deep learning (2014, methodology): “For example, the deep-structured CRF, which stacks many layers of CRFs, have been successfully used in the task of language identification [103], phone recognition [102], sequential labeling in natural language processing [96], and confidence calibration in speech recognition [178, 179].”
Selective Classification Can Magnify Disparities Across Groups (2020, background): “Prior work has shown that, across a broad array of applications, more confident predictions tend to be more accurate (Hanczar & Dougherty, 2008; Yu et al., 2011; Toplak et al., 2014; Mozannar & Sontag, 2020; Kamath et al., 2020).”
Calibrated Structured Prediction (2015, background): “Its importance has been discussed and emphasized in medicine [22], natural language processing [23], speech recognition [21], meteorology [10], econometrics [9], and psychology [24].”
Active and Semi-Supervised Learning in ASR: Benefits on the Acoustic and Language Models (2016, methodology): “A survey of possible confidence measures is given in [8] and several techniques for confidence score calibration have been developed in [9].”
Predicting speech recognition confidence using deep learning with word identity and score features (2013, methodology): “A number of approaches have been proposed in the area of confidence evaluation and can be categorized in 3 groups: (1) confidence measure as combination of predictor features: a two-class classifier is trained to determine confidences based on predictors dumped from an ASR engine [1], (2) confidence”
A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks (2016, result): “This is consistent with a great deal of anecdotal evidence from researchers (Nguyen & O’Connor, 2015; Yu et al., 2010; Provost et al., 1998; Nguyen et al., 2015).”
Principled Detection of Out-of-Distribution Examples in Neural Networks (2017, methodology): “Using a large temperature drives the softmax score closer to 1/N , as observed in [19, 23, 24].”
An Overview of Deep-Structured Learning for Information Processing (2011, methodology): “Many of the discriminative techniques in signal and information processing apply to shallow architectures such as HMMs (e.g., He et al., 2008; Jiang and Li, 2010; Xiao and Deng, 2010; Gibson and Hain, 2010) or CRFs (e.g., Yang and Furui, 2009; Yu, Li, and Deng, 2010; Hifny and Renals, 2009).”
Exploiting deep neural networks for detection-based speech recognition (2013, methodology): “This search technique, known as the top-down integrated search strategy, has attained remarkable results in many ASR tasks.”