Hypothesis spaces for minimum Bayes risk training in large vocabulary speech recognition

By Matthew Gibson, Thomas Hain. 2006

136 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Speech & Audio

In Signal Processing

Citations in the corpus

RWTH ASR Systems for LibriSpeech: Hybrid vs Attention - w/o Data Augmentation (2019, methodology): “Sequence discriminative training is performed using a latticebased version of the state-level minimum Bayes risk (sMBR) criterion [19].”
Wav2Letter: an End-to-End ConvNet-based Speech Recognition System (2016, methodology): “n the context of hybrid HMM/NN systems, such as the MMI criterion [2] which maximizes the mutual information between the acoustic sequence and word sequences or the Minimum Bayse Risk (MBR) criterion [5]. More recently, standalone neural network architectures have been trained using criterions which”
Parallel training of Deep Neural Networks with Natural Gradient and Parameter Averaging (2014, methodology): “We generally use state-level Minimum Bayes Risk (sMBR) [27, 28] although we have also implemented Minimum Phone Error (MPE) [26] and Boosted MMI [25].”
Letter-Based Speech Recognition with Gated ConvNets (2017, background): “The MMI criterion (Bahl et al., 1986) maximizes the mutual information between the acoustic sequence and word sequences or the Minimum Bayes Risk (MBR) criterion (Gibson and Hain, 2006).”
Semi-supervised maximum mutual information training of deep neural network acoustic models (2015, methodology): “A number of discriminative criteria such as MMI [1], MCE[2], MPE [3], sMBR [4, 5] and bMMI [6] have been developed and used in HMM-based speech recognition [7, 8, 9, 10].”
Word-level Speech Recognition with a Dynamic Lexicon (2019, methodology): “Discriminative loss functions like Minimum Bayes Risk training can be used to optimize the WER directly but are not end-to-end [13].”
Sequence-discriminative training of deep neural networks (2013, methodology): “Like before, we get: ∂FMBR ∂aut(s) = κγMBRut (s).”
Knowledge distillation for small-footprint highway networks (2016, background): “,xT } and Y as the sequence of labels, where T is the length of the signal, the loss function from the state-level minimum Bayesian risk criterion(sMBR) [19, 20] is defined as”
LSTM Language Models for LVCSR in First-Pass Decoding and Lattice-Rescoring (2019, methodology): “The acoustic model is a multilayer BLSTM neural network trained with the state-level minimum Bayes Risk (sMBR) criterion [30].”
Word-Level Speech Recognition With a Letter to Word Encoder (2019, methodology): “Discriminative loss functions like Minimum Bayes Risk training can be used to optimize the WER directly but are not end-to-end (Gibson & Hain, 2006).”
A Fully Differentiable Beam Search Decoder (2019, methodology): “…Jaitly, 2014) and the Auto Segmentation (ASG) criterion (Collobert et al., 2016), as well as Minimum Bayes Risk (MBR and sMBR) (Goel and Byrne, 2000; Gibson and Hain, 2006; Sak et al., 2015; Prabhavalkar et al., 2018) and the Maximum Mutual Information (MMI) criterion (Bahl et al., 1986), amongst…”
Boosted MMI for model and feature-space discriminative training (2008, background): “We also recently investigated [7] replacing A(s, sr) with various other functions including a state-level accuracy which we call state-level Minimum Bayes Risk (s-MBR) after [8].”
The Rwth Asr System for Ted-Lium Release 2: Improving Hybrid Hmm With Specaugment (2020, methodology): “Additionally, speaker adaptive training (SAT) using i-vectors and sequence discriminative training, such as the state-level minimum Bayes risk (sMBR) [5] criterion, are often applied for further improvements.”
Sequence Student-Teacher Training of Deep Neural Networks (2016, background): “Training with the state-level loss function is known as the state-level MBR (sMBR) criterion [19, 20].”
A novel pyramidal-FSMN architecture with lattice-free MMI for speech recognition (2018, background): “Since speech recognition is a sequential problem, several sequential discriminative training criteria are applied after CE training such as maximum mutual information (MMI) [16], minimum Bayes risk (MBR) [17] and minimum phone error (MPE) [18].”
Lattice-based optimization of sequence classification criteria for neural-network acoustic modeling (2009, background): “Typically, the labels are “hard,” ŷrt(i) ∈ {0, 1}, i = 1, · · · , N , and are defined either by manual labeling (e.g., TIMIT) or through forced alignment.”
Extracting deep neural network bottleneck features using low-rank matrix factorization (2014, methodology): “After speaker-adaptated training (SAT) and minimum Bayes risk (MBR) discriminative training [21] on the LrSBN features, the performance of the LrSBN system is similar to the hybrid DNN system with sequence training (SQ) in Bengali, while performing 0.”
Improving deep neural network acoustic models using generalized maxout networks (2014, methodology): “This implementation supports Restricted Boltzmann Machines (RBM) pretraining [1, 2, 3], stochastic gradient descent training using NVidia graphics processing units (GPUs), and discriminative training such as boosted MMI [16] and state-level minimum Bayes risk (sMBR) [17, 18].”
Semi-supervised training of Deep Neural Networks (2013, methodology): “…Machines (RBM) by Contrastive Divergence algorithm, 2. frame-classiﬁcation training [11][12] – mini-batch Stochastic Gradient Descent (SGD), optimizing frame cross-entropy, 3. sequence-discriminative training [13][14][15] – SGD with per-sentence updates, optimizing state Minimum Bayes Risk (sMBR).”
Error back propagation for sequence training of Context-Dependent Deep NetworkS for conversational speech transcription (2013, background): “This is also known as the cross-entropy (CE) criterion (with Kronecker delta δ): FCE = ∑ r ∑ t logP (ŝr(t)|or(t)), (2) eCEs (r, t) = δs,ŝr(t) − P (s|or(t)) (3)”