Investigation of transfer learning for ASR using LF-MMI trained neural networks

By Pegah Ghahremani, Vimal Manohar, Hossein Hadian, Daniel Povey, Sanjeev Khudanpur. 2017

75 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Ocean Engineering, Representation & Self-Supervised Learning, Speech & Audio

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jinyu Li
2. Tara N. Sainath
3. Gabriel Synnaeve
4. Yoshua Bengio
5. Ronan Collobert
6. Dong Yu
7. Yu Zhang
8. Michael Auli

In Ocean Engineering

1. Sanjeev Khudanpur
2. Daniel Povey
3. Vimal Manohar
4. Yoshua Bengio
5. Gabriel Synnaeve

In Representation & Self-Supervised Learning

1. Daniel Povey
2. Yoshua Bengio
3. Sanjeev Khudanpur
4. Vimal Manohar
5. Ronan Collobert
6. Mirco Ravanelli
7. Philipp Koehn
8. Hossein Hadian

In Speech & Audio

1. Jinyu Li
2. Gabriel Synnaeve
3. Tara N. Sainath
4. Ronan Collobert
5. Chung-Cheng Chiu
6. Yu Zhang
7. Qiantong Xu
8. Wei-Ning Hsu

Citations in the corpus

• Rethinking Evaluation in ASR: Are Our Models Robust Enough? (2020, background): “Previous works that study transfer in ASR include [1] that studied transferring varying number of layers trained out-ofdomain, from SwitchBoard to AMI-IHM or from LibriSpeech to”
• Toward Domain-Invariant Speech Recognition via Large Scale Training (2018, methodology): “Unlike [7], the current work studies domain robustness in a much larger scale, where data sparsity is not necessarily a challenge.”
• vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations (2019, neutral): “8† Supervised transfer-learning (Ghahremani et al., 2017) - 4.”
• wav2vec: Unsupervised Pre-training for Speech Recognition (2019, neutral): “8† Supervised transfer-learning (Ghahremani et al., 2017) - 4.”
• JHU Kaldi system for Arabic MGB-3 ASR challenge using diarization, audio-transcript alignment and transfer learning (2017, methodology): “We use a weight transfer approach suggested in [8].”
• Recognizing Long-Form Speech Using Streaming End-to-End Models (2019, methodology): “As can be seen in the table, for E2E ASR models, we find that, contrary to [31], the best strategy is to sample utterances proportional to the amount of training data in each domain.”
• Speech Model Pre-training for End-to-End Spoken Language Understanding (2019, background): “The computer vision [15, 16], natural language processing [17–21], and ASR [22–25] communities have attacked the problem of limited supervised training data with great success by pre-training deep models on related tasks for which there is more training data.”
• Speech Recognition (2018, unknown): “8† Supervised transfer-learning (Ghahremani et al., 2017) - 4.”
• Semi-Supervised Training of Acoustic Models Using Lattice-Free MMI (2018, methodology): “The phone LM used for creating the denominator FST is estimated using phone sequences from both supervised and unsupervised data as in [28, 29].”
• Freezing Subnetworks to Analyze Domain Adaptation in Neural Machine Translation (2018, background): “We interpret these gains as trivial (and possibly the result of variance) but there may be an NMT continued training scenario in which freezing could increase performance by acting as a regularizer (see Ghahremani et al., 2017).”