Semi-supervised maximum mutual information training of deep neural network acoustic models

By Vimal Manohar, Daniel Povey, Sanjeev Khudanpur. 2015

49 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

In Speech & Audio

In Signal Processing

Citations in the corpus

Iterative Pseudo-Labeling for Speech Recognition (2020, neutral): “Semi-supervision in ASR is well-studied [15, 16, 17, 18, 19, 20]; our work builds primarily on recent work with end-to-end systems, especially PL [9, 1].”
Lessons from Building Acoustic Models with a Million Hours of Speech (2019, background): “A further challenge is applying sequence discriminative training, where label errors have a larger detrimental effect ([24], [25]).”
Semi-Supervised Training of Acoustic Models Using Lattice-Free MMI (2018, methodology): “Lattice supervision has been used in [15] with conventional sequence objectives like MMI [16] as well as in [17, 18] for semi-supervised training with lattice entropy minimization.”
Semi-supervised ensemble DNN acoustic model training (2017, background): “confidence data as different resources, a multi-task training architecture inspired from multi-lingual modeling [10, 11, 12, 13, 14, 15] can be applied for semi-supervised training.”
Semi-Supervised DNN Training with Word Selection for ASR (2017, methodology): “In [12], the untranscribed data are used even for the sequence-discriminative training.”
Large scale weakly and semi-supervised learning for low-resource video ASR (2020, methodology): “Incorporating the LM information within a sequence discriminative loss function for self-labeling was proven to be a difficult task given the noise in labels [46, 48, 38].”
slimIPL: Language-Model-Free Iterative Pseudo-Labeling (2020, methodology): “ and web-scale bitext mining [44]; • automatic speech recognition (ASR): representation learning [45, 2, 46, 47], local prior matching [4], adversarial training [48], back-translation [49] and others [50, 51, 52, 53, 54]. Below, we give an overview of the approaches in ASR that are most recent and r”
A time delay neural network architecture for efficient modeling of long temporal contexts (2015, methodology): “We found that the method of using the mean posterior (computed over a subset of the training data) as the prior [22] gave an improved performance when decoding with sMBR tuned models, so we used this method.”
On the limit of English conversational speech recognition (2021, background): “Unlike the hybrid approach where the framewise priors can even directly be estimated and integrated into the softmax layer [39], it is difficult to estimate the sequence prior p(w 1 ).”
Active and Semi-Supervised Learning in ASR: Benefits on the Acoustic and Language Models (2016, methodology): “For sequence-discriminative bMMI training, it is known that possible errors in the transcripts can have a dramatic negative influence on the quality of the resulting AM [13].”
The RWTH/UPB/FORTH System Combination for the 4th CHiME Challenge Evaluation (2016, methodology): “Table 9 shows the performance gain obtained by the sequence-discriminative training of the BLSTM [31].”
JHU ASpIRE system: Robust LVCSR with TDNNS, iVector adaptation and RNN-LMS (2015, methodology): “We found that the method of using the mean posterior (computed over a subset of the training data) as the prior [22] gave an improved performance when decoding with sMBR-trained models, so we used this method.”