Maximum mutual information estimation of hidden Markov model parameters for speech recognition

By L. Bahl, Peter F. Brown, P. D. Souza, Robert L. Mercer. 1986

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Speech & Audio

Citations in the corpus

A Diversity-Promoting Objective Function for Neural Conversation Models (2015, methodology): “We propose to capture this intuition by using Max-1 imum Mutual Information (MMI), ﬁrst introduced in speech recognition (Bahl et al., 1986; Brown, 1987), as an optimization objective that measures the mutual dependence between inputs and outputs.”
Unsupervised Speech Recognition (2021, background): “…with a maximal likelihood objective (Bahl et al., 1983; Juang et al., 1986): p∗LM = arg maxpLM N∑ i=1 log pLM (Yi) p ∗ AM = arg maxpAM N∑ i=1 log pAM (Xi | fPM (Yi)), but alternative discriminative objectives for AMs taking LMs into account have also been explored (Bahl et al., 1986; Povey, 2005).”
Wav2Letter: an End-to-End ConvNet-based Speech Recognition System (2016, neutral): “Other alternatives have been explored in the context of hybrid HMM/NN systems, such as the MMI criterion [2] which maximizes the mutual information between the acoustic sequence and word sequences or the Minimum Bayse Risk (MBR) criterion [5].”
Parallel training of Deep Neural Networks with Natural Gradient and Parameter Averaging (2014, background): “There are two popular classes of sequence/discriminative objective functions: • Maximum Mutual Information (MMI)-like objec-tive functions [24, 23], more properly called conditional maximum likelihood: these have the form of sum over all utterances of the log-posterior of the correct word sequence…”
Feature Learning in Deep Neural Networks - A Study on Speech Recognition Tasks (2013, methodology): “The model parameters can be discriminatively trained using an objective function such as maximum mutual information (MMI) [1] or minimum phone error rate (MPE) [2].”
A Survey on Dialogue Systems: Recent Advances and New Frontiers (2017, methodology): “models assign high probability to “safe responses when optimizing the likelihood of outputs given inputs. They used a Maximum Mutual Information (MMI), which was ﬁrst introduced in speech recognition [6; 9], as an optimization objective. It measured the mutual dependence between inputs and outputs, ”
A novel pyramidal-FSMN architecture with lattice-free MMI for speech recognition (2018, background): “Since speech recognition is a sequential problem, several sequential discriminative training criteria are applied after CE training such as maximum mutual information (MMI) [16], minimum Bayes risk (MBR) [17] and minimum phone error (MPE) [18].”
A Systematic Review of Hidden Markov Models and Their Applications (2020, methodology): “Figure 7 represents various applications of first-order HMM. Bahl et al. [11] described a method for estimating the maximum mutual information for various parameters of HMM in speech recognition.”
Hidden Markov Models for Speech Recognition (1991, methodology): “Previous attempts (Bahl et al. 1986; Brown 1987) at using the MMI criterion have not produced an estimation procedure that is guaranteed to converge to an optimal solution either.”
Scalable Minimum Bayes Risk Training of Deep Neural Network Acoustic Models Using Distributed Hessian-free Optimization (2012, background): “We compare speaker-independent (SI) and speaker-adaptive (SA) GMM models with either maximum-likelihood (ML), modelspace BMMI (BMMI), or feature- and model-space BMMI (FMMI+BMMI) training to DBN models trained using either cross-entropy optimized using stochastic gradient descent or sMBR optimized u”
Adaptive Multimodal Fusion by Uncertainty Compensation With Application to Audiovisual Speech Recognition (2009, methodology): “The importance of discriminative approaches to audio-only ASR has been highlighted by the success of discriminative model training techniques using the maximum mutual information [35] or the minimum classification error rate”
A Tutorial on Energy-Based Learning (2006, methodology): “Discriminative learning methods were proposed to train HMM-based speech recognition systems in the late 1980’s [3, 46].”
Towards End-To-End Speech Recognition with Recurrent Neural Networks (2014, methodology): “Full-sequence training methods such as Maximum Mutual Information have been used to directly train HMM-neural network hybrids to maximise the probability of the correct transcription (Bahl et al., 1986; Jaitly et al., 2012).”
Boosted MMI for model and feature-space discriminative training (2008, background): “The Maximum Mutual Information (MMI) objective function [3, 4, 5] seeks to maximize the posterior probability of the correct utter-”
Maximizing Mutual Information for Tacotron (2019, methodology): “Maximum mutual information is used to estimate HMMs in speech recognition [2].”