Adversarial Multi-Task Learning of Deep Neural Networks for Robust Speech Recognition

By Yusuke Shinohara. 2016

169 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jinyu Li
2. Dong Yu
3. Yoshua Bengio
4. James R. Glass
5. Shinji Watanabe
6. Li Deng
7. Wei-Ning Hsu
8. Gabriel Synnaeve

In Computer Vision and Pattern Recognition

1. James R. Glass
2. Dong Yu
3. Dacheng Tao
4. Luc Van Gool
5. Wei-Ning Hsu
6. Xiaogang Wang
7. Andrew Zisserman
8. Yoshua Bengio

In Safety, Robustness & Interpretability

1. Arun Narayanan
2. Shinji Watanabe
3. Jinyu Li
4. Khe Chai Sim
5. James R. Glass
6. Wei-Ning Hsu
7. Yifan Gong
8. Zhong Meng

In Speech & Audio

1. Jinyu Li
2. Wei-Ning Hsu
3. Shinji Watanabe
4. James R. Glass
5. Yu Zhang
6. Gabriel Synnaeve
7. Yoshua Bengio
8. Ronan Collobert

In Signal Processing

1. Jinyu Li
2. Dong Yu
3. Shinji Watanabe
4. Wei-Ning Hsu
5. James R. Glass
6. Yoshua Bengio
7. Tara N. Sainath
8. Li Deng

Citations in the corpus

• Toward Domain-Invariant Speech Recognition via Large Scale Training (2018, methodology): “We also expect modeling strategies like domain adversarial training [18, 37, 38] and better training techniques [39] to improve performance.”
• A Survey on Multi-Task Learning (2017, background): “Building on the first category, the second category is to use adversarial learning, which is inspired by generative adversarial networks, to learn a common feature representation for MTL as did in [20], [21].”
• Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Data (2017, background): “Learning to disentangle the mixed information from the surface representation is essential for a wide variety of speech applications: for example, noise robust speech recognition [41, 38, 37, 16], speaker verification [5], and voice conversion [40, 29, 24].”
• English Conversational Telephone Speech Recognition by Humans and Machines (2017, methodology): “Among the five LSTM-based and convolution-based LMs, word-LSTM-MTL achieved the best WER of 5.6% and 10.3% on SWB and CH respectively.”
• Cross-Lingual Transfer Learning for Question Answering (2019, methodology): “It has been successfully applied in computer vision (Ganin et al., 2016) and speech recognition (Shinohara, 2016).”
• Massively Multilingual Adversarial Speech Recognition (2019, background): “Domain-adversarial training has been used in speech recognition to learn features invariant to noise conditions (Shinohara, 2016), accents (Sun, 2018), and sex (Tripathi et al., 2018).”
• Cycle-Consistent Speech Enhancement (2018, methodology): “In speech area, it has been applied to speech enhancement [24, 25, 26, 27], voice conversion [28, 29], acoustic model adaptation [30, 31, 32], noise-robust [33, 34] and speaker-invariant [35, 36] ASR using gradient reversal layer (GRL) [37].”
• Unsupervised Adaptation of Acoustic Models for ASR Using Utterance-Level Embeddings from Squeeze and Excitation Networks (2019, methodology): “Such networks have been used for achieving speaker invariance [24, 25, 26], noise invariance [27, 28], etc.”
• Speaker Invariant Feature Extraction for Zero-Resource Languages with Adversarial Learning (2018, methodology): “In [14], adversarial loss was inserted into a senone classification network in an attempt to make the original classification network robust against specific variances.”
• Recent progresses in deep learning based acoustic models (2017, background): “Recently, the concept of adversarial training [147] was explored for noise-robust ASR [148]–[155] and speaker invariant training [156], [157].”
• Joint Masked CPC And CTC Training For ASR (2020, methodology): “As our approach addresses both, a contrastive learning task and speech recognition task, this paper is related to the field of multi-task learning [14, 15].”
• A Survey of Unsupervised Deep Domain Adaptation (2018, methodology): “For time-series data, domain adaptation has been used for learning temporal latent relationships in health data across different population age groups [189], to perform speech recognition [100, 218, 282], for predicting driving maneuvers [238], anomaly detection [242], and inertial tracking [35].”
• Invariant Representations for Noisy Speech Recognition (2016, background): “Recently, in Shunohara (2016) a multilayer sigmoidal network is trained in an adversarial fashion on an in-house transcription task corrupted by noise.”
• Unsupervised adaptation with domain separation networks for robust speech recognition (2017, methodology): “This GRL approach has been applied to acoustic models for unsupervised adaptation in [25] and for increasing noise robustness in [26, 27].”
• Speaker-Invariant Training Via Adversarial Learning (2018, methodology): “In speech, it has been applied to noise-robust [15, 16, 17, 18, 19] and conversational ASR [20] using gradient reversal layer [21] or domain separation network [22].”