UTILIZING UNTRANSCRIBED TRAINING DATA TO IMPROVE PERFORMANCE

By G. Zavaliagkos, Thomas Colthurst. 1998

Subfields: Natural Language Processing, Speech & Audio

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Speech & Audio

Citations in the corpus

BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition (2021, methodology): “There is extensive literature on pre-training [1]–[13] and self-training [14]–[25] in ASR, a subset of which we list in the bibliography.”
Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition (2020, methodology): “Self-training has been a popular technique in ASR studied extensively in the literature [25–33].”
Improved Noisy Student Training for Automatic Speech Recognition (2020, background): “Self-training has a long history of being applied to ASR through numerous studies, e.g., [12, 13, 14, 15, 16, 17, 18, 19].”
w2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Training (2021, positive): “Self-training has been a practically useful and extensively studied technique in ASR [4, 5, 6, 7, 8, 9].”
Large scale weakly and semi-supervised learning for low-resource video ASR (2020, methodology): “Reducing the amount of manual transcription for building speech recognition systems has been a constant research theme over the past three decades [40, 20, 9, 5, 41, 42] given the revolutionary increase and access to unlabeled data and computational resources.”
Unsupervised training of acoustic models for large vocabulary continuous speech recognition (2005, background): “In [1], the authors investigate such a scenario for the CallHome corpus.”
Data augmentation for low resource languages (2014, methodology): “The unsupervised data may be adopted by recognising it with an existing or boot-strapped system, filtering out those utterances that fail to decode/pass confidence threshold [15, 16] and re-training the system on su-”
Lightly Supervised Acoustic Model Training (2000, background): “We show that this approach can dramatically reduces the cost of building acoustic models.”
Unsupervised versus supervised training of acoustic models (2008, background): “A great amount of work has been reported on unsupervised training [1,2,3,4,5,6].”
Contrastive Semi-Supervised Learning for ASR (2021, methodology): “Since the 1990s, pseudo-labeling showed improvements to classical HMM/GMM systems as well as hybrid-NN systems [2, 3, 4, 6, 5, 7], and also for end-to-end ASR approaches [8, 9, 10, 11, 12].”
Lightly supervised and unsupervised acoustic model training (2002, methodology): “One of the most often cited costs in developmentis that of obtaining this necessary transcribed acoustic training data, which is1”
Hubert: How Much Can a Bad Teacher Benefit ASR Pre-Training? (2021, methodology): “Pseudo-labeling [12, 13, 14, 15] can be thought of as a special instance of this approach in which the target sub-network is trained using supervised data, then used to generate pseudo-labels for a larger volume of unlabeled audio recordings.”