Boosted MMI for model and feature-space discriminative training

By Daniel Povey, Dimitri Kanevsky, Brian Kingsbury, Bhuvana Ramabhadran, George Saon, Karthik Venkat Ramanan. 2008

438 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Dong Yu
2. Li Deng
3. Jinyu Li
4. Yoshua Bengio
5. Tara N. Sainath
6. Jianfeng Gao
7. Quoc V. Le
8. Shinji Watanabe

In Signal Processing

1. Dong Yu
2. Jinyu Li
3. Li Deng
4. Tara N. Sainath
5. Yifan Gong
6. S. Renals
7. Shinji Watanabe
8. Daniel Povey

Citations in the corpus

• English Conversational Telephone Speech Recognition by Humans and Machines (2017, methodology): “The training consists of 14 passes of cross-entropy followed by 1 pass of SGD sequence training using the boosted MMI criterion [16] smoothed by adding the scaled gradient of the cross-entropy loss [17].”
• A pitch extraction algorithm tuned for automatic speech recognition (2014, methodology): “Our current recipe is based on combination of three types of Kaldi system, but for speed we only test one of them here: it is a Subspace Gaussian Mixture Model (SGMM) system [12], discriminatively trained with boosted MMI (bMMI) [13].”
• Unsupervised Testing Strategies for ASR (2011, methodology): “Specifically we use a PLP front-end [11] together with LDA and STC [12], and optimize our acoustic models using BMMI [13] on mostly un-supervised data mixed from both tasks.”
• Sequence-discriminative training of deep neural networks (2013, methodology): “In Table 1, we compare the results of the maximum likelihood (ML) trained models with those trained using BMMI with a boosting factor b = 0.1 (cf. equation (10)).”
• Hybrid acoustic models for distant and multichannel large vocabulary speech recognition (2013, methodology): “The GMM-HMM systems are trained on the speaker adapted LDA+STC features for the IHM case, or on the unadapted features for the SDM/MDM case, using the boosted maximum mutual information (BMMI) [26] criterion.”
• Deep Belief Networks using discriminative features for phone recognition (2011, methodology): “Significant word error rate reductions were achieved by fine-tuning the ML trained GMMs using discriminative training criteria [3, 4].”
• Building DNN acoustic models for large vocabulary speech recognition (2014, methodology): “Discriminative loss functions were initially developed for GMM acoustic models [38], [39], [40], [41], but were recently applied to DNN acoustic model training [4], [3], [42].”
• Application of Pretrained Deep Neural Networks to Large Vocabulary Speech Recognition (2012, methodology): “The acoustic models were further improved with BMMI [13].”
• Optimization Algorithms and Applications for Speech and Language Processing (2013, methodology): “…approach is to use an approximation to the steepest descent direction , and define (9) In unconstrained optimization, this choice of will yield a direction of ascent for provided that (10) In other words, for all sufficiently small choices of in (4), we will identify a new iterate such that .”
• “Your Word is my Command”: Google Search by Voice: A Case Study (2010, methodology): “We optimize ML, MMI, and ’boosted’-MMI [8] objective functions in training.”
• A time delay neural network architecture for efficient modeling of long temporal contexts (2015, methodology): “However, speaker adaptation techniques like fMLLR [4] require two passes of decoding.”
• Convolutional Neural Networks for Distant Speech Recognition (2014, methodology): “In each case we compare the CNN to two baseline systems: (1) a Gaussian mixture model (GMM) system, discriminatively trained using boosted maximum mutual information (BMMI) [39], with mel-frequency cepstral coefficient (MFCC) features”
• Softmax-Margin CRFs: Training Log-Linear Models with Cost Functions (2010, methodology): “6 and is a generalization of that used by Povey et al. (2008) and similar to that used by Sha and Saul (2006).”
• Integrating Deep Neural Networks into Structural Classification Approach based on Weighted Finite-State Transducers (2012, methodology): “The first objective function is a boosted MMI function [13] combined with a minimum transition error (MTE) measure, as follows:”
• Unsupervised cross-lingual knowledge transfer in DNN-based LVCSR (2012, methodology): “We compare the hybrid setup to a HMM-GMM system that uses both model and feature-space discriminative training using boosted maximum mutual information (BMMI) estimation [26], referred to as fBMMI+BMMI in Table 1.”
• Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition (2012, background): “Recently, discriminative training strategies such as MMI [5], MCE [6], [7], MPE [8], [9], and large-margin techniques [10]–[17] have been proposed.”