Unsupervised versus supervised training of acoustic models

By Jeff Z. Ma, Richard M. Schwartz. 2008

57 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

In Speech & Audio

In Signal Processing

Citations in the corpus

Iterative Pseudo-Labeling for Speech Recognition (2020, neutral): “Semi-supervision in ASR is well-studied [15, 16, 17, 18, 19, 20]; our work builds primarily on recent work with end-to-end systems, especially PL [9, 1].”
Self-Training for End-to-End Speech Recognition (2019, background): “In addition to WER, we report WER recovery rate (WRR) [11] to demonstrate how much gap between the baseline and the oracle that we can bridge with pseudo-labels.”
slimIPL: Language-Model-Free Iterative Pseudo-Labeling (2020, methodology): “ and web-scale bitext mining [44]; • automatic speech recognition (ASR): representation learning [45, 2, 46, 47], local prior matching [4], adversarial training [48], back-translation [49] and others [50, 51, 52, 53, 54]. Below, we give an overview of the approaches in ASR that are most recent and r”
Unsupervised Testing Strategies for ASR (2011, background): “While some published results considered unsupervised maximum likelihood estimation of model parameters [5], many systems use unsupervised discriminative optimization, directly us-ing recognizer output as input [6].”
Analysis of low-resource acoustic model self-training (2009, methodology): “We report results using one resource condition from Ma and Schwartz (2008).”
Semi-supervised Training for End-to-end Models via Weak Distillation (2019, methodology): “Confidence scores from an existing ASR system is commonly used to select unsupervised data for training with more data [11–15].”
Semi-Supervised Training of Acoustic Models Using Lattice-Free MMI (2018, methodology): “We use WER Recovery Rate (WRR) [25] as a metric to evaluate the WER improvements from semi-supervised training: WRR = BaselineWER− SemisupWER BaselineWER−OracleWER .”
Unsupervised training of an HMM-based self-organizing unit recognizer with applications to topic classification and keyword discovery (2014, methodology): “If some transcribed audio is available, semi-supervised raining (Lamel et al., 2002; Zavaliagkos et al., 1998; Ma and Schwartz, 2008; Li et al., 2007) can be used, which s similar to our unsupervised approach with the exception that the model would be initialized using a small set of ranscribed…”
A big data approach to acoustic model training corpus selection (2014, methodology): “Rather than relying on small supervised training sets, one can instead exploit the large amount of audio data that is processed by systems like Voice Search, together with semi-supervised techniques to construct large training sets with automatically derived transcripts [4, 5, 6, 7, 8].”