Some statistical issues in the comparison of speech recognition algorithms

By Laurence S. Gillick, Stephen J. Cox. 1989

Subfields: Statistics and Probability, Speech & Audio

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

In Speech & Audio

Citations in the corpus

Light Gated Recurrent Units for Speech Recognition (2018, methodology): “This improvement is statistically signiﬁcant according to the Matched Pairs Sentence Segment Word Error Test (MPSSW) [69], that is conducted with NIST sclite sc sta t tool with a p-value of 0 .”
Continual Learning for Named Entity Recognition (2021, background): “The goal of CL is to gradually extend the knowledge acquired in a model (Lange et al. 2019), without incurring any Catastrophic Forgetting (CF) (Goodfellow et al. 2013), mainly to account for changes in the data distribution.”
Domain Adaptation with Structural Correspondence Learning (2006, neutral): “(Gillick and Cox, 1989).”
Shallow Parsing with Conditional Random Fields (2003, methodology): “One 2Although L-BFGS has a slightly higher penalized loglikelihood, its log-likelihood on the data is actually lower than that of preconditioned CG and mixed CG training. such test is McNemar test on paired observations (Gillick and Cox, 1989).”
Speech Recognition via Large-Scale Weak Supervision (unknown): “The standard statistical tests for determining if two word error rates are different is the Matched-Pair Sentence Segment Word Error (MAPSSWE) test, introduced in Gillick and Cox (1989).”
Adaptive Multimodal Fusion by Uncertainty Compensation With Application to Audiovisual Speech Recognition (2009, methodology): “The audiovisual experiments reported next use the best-performing six-shape/six-texture visemic AAM visual feature set.”
Rapid speaker adaptation in eigenvoice space (2000, methodology): “[16] was applied to the null hypothesis: that the difference between MLED.”
Feature Selection via Coalitional Game Theory (2007, methodology): “The McNemmar test (Gillick & Cox, 1989) on the results summarized in Table 3 identiﬁed no signiﬁcant superiority of any feature selection method on any of the data sets.”
Multi-domain spoken language understanding with transfer learning (2009, methodology): “The significance of our results was determined using the McNemar paired test for proportions (Gillick and Cox, 1989), which uses individual labeling decisions to compare the correctness of two models.”
A fast maximum likelihood nonlinear feature transformation method for GMM-HMM speaker adaptation (2014, background): “0% over the fMLLR with bias compensation system and is statistically significant with significance level of 9:5 10 10 [25].”
Neural Models of Text Normalization for Speech Applications (2019, unknown): “05), according to the McNemar test (Gillick and Cox 1989).”
Pivot Based Language Modeling for Improved Neural Domain Adaptation (2018, background): “McNemar paired test for labeling disagreements ((Gillick and Cox, 1989; Blitzer et al., 2006), p < 0.”
Voting Between Multiple Data Representations for Text Chunking (2005, methodology): “With this distribution, we can directly compute a 2-tailed P value based on the following formula defined in [ GC89 ],”
Modeling Latent-Dynamic in Shallow Parsing: A Latent Conditional Model with Improved Inference (2008, methodology): “One such test is the McNemar test on paired observations ( Gillick & Cox 1989 ).”
Efficient Inference in Large Conditional Random Fields (2006, background): “1% [8]), indicating that the tying of features is smoothing the model.”
An experimental study on joint modeling of mixed-bandwidth data via deep neural networks for robust speech recognition (2016, methodology): “Here we adopt a “matched pair test” in [22], [23], which is a two-tailed test with the null hypothesis that there is no performance difference between the two systems.”
Weakly Supervised Learning with Multi-Stream CNN-LSTM-HMMs to Discover Sequential Parallelism in Sign Language Videos (2020, methodology): “The MAPSSWE test was suggested for speech recognition evaluations by Gillick [70].”