Speech enhancement and recognition using multi-task learning of long short-term memory recurrent neural networks

By Zhuo Chen, Shinji Watanabe, Hakan Erdogan, John R. Hershey. 2015

157 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yoshua Bengio
2. Jinyu Li
3. Dong Yu
4. Tara N. Sainath
5. Shinji Watanabe
6. Bo Li
7. James R. Glass
8. Mirco Ravanelli

In Speech & Audio

1. Yoshua Bengio
2. Mirco Ravanelli
3. Jinyu Li
4. Tara N. Sainath
5. Bo Li
6. Shinji Watanabe
7. Hung-yi Lee
8. Zhuo Chen

In Signal Processing

1. Jinyu Li
2. Yoshua Bengio
3. Mirco Ravanelli
4. Dong Yu
5. Tara N. Sainath
6. Shinji Watanabe
7. Nima Mesgarani
8. Bo Li

Citations in the corpus

• A Survey on Multi-Task Learning (2017, methodology): “Feature Transformation Visual tracking [192], [193] Protein subcellular location prediction [194] Speech synthesis [195], [196] Learning to rank [197] Stock prediction [198] Action recognition [199] Protein interaction prediction [200] Speech recognition [201], [202] Localization [203] Facial landma”
• Deep Learning for Environmentally Robust Speech Recognition (2017, background): “76 (SDR) [63] mapping: log mag hybrid DNN ≈25% [110] masking: IRM (SA; log Mel) MTL; hybrid BLSTM 16.”
• Continual Learning in Automatic Speech Recognition (2020, methodology): “An offline approach to overcome this problem is to accumulate the training data from new domains together with the original training data, and re-train the whole classifier using multitask[1, 2], multi-condition[3, 4] or data-augmentation[5, 6] techniques.”
• Modeling Prosodic Phrasing With Multi-Task Learning in Tacotron-Based TTS (2020, methodology): “MTL has been widely used in speech enhancement [17], and speech recognition [18].”
• Batch-normalized joint training for DNN-based distant speech recognition (2016, background): “Although joint training for speech recognition is still an under-explored research direction, such a paradigm is progressively gaining more attention and some interesting works in the field have been recently published [29, 30, 31, 32, 33, 34, 35].”
• Human Listening and Live Captioning: Multi-Task Training for Speech Enhancement (2021, methodology): “ASR Multi-task Training: There are prior studies that employ joint or multi-task training for the front-end and backend [12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22].”
• SpeechNet: A Universal Modularized Model for Speech Processing Tasks (2021, methodology): “(Chen et al., 2015) uses MTL to solve SE and ASR.”
• Bone-conducted speech enhancement using deep denoising autoencoder (2018, background): “, 2015), long short-term memory (Chen et al., 2015; Weninger et al., 2015; Sun et al., 2017), convolutional neural networks (Fu et al.”
• A network of deep neural networks for Distant Speech Recognition (2017, methodology): “Similarly to this work, an iterative pipeline based on feeding the speech recognition output into a speech enhancement DNN has recently been proposed in [19, 30].”
• DEEP ATTRACTOR NETWORK FOR SINGLE-MICROPHONE SPEAKER SEPARATION (2016, background): “On the other hand, when the attractor vectors are considered as free parameters in the network, DANet reduces to a classification network [4, 5], and Equation 1 becomes a fully-connected layer.”
• Speaker-Independent Speech Separation With Deep Attractor Network (2017, methodology): “On the other hand, when the attractor points are considered as a layer in the network instead of dynamically formed by the embeddings, DANet reduces to a classification network [9], [6], and Eqn.”
• Multichannel Signal Processing With Deep Neural Networks for Automatic Speech Recognition (2017, background): “3) Regularization with MTL: Multi-task learning has been shown to yield improved robustness [26], [38], [39].”
• Improving Speech Recognition by Revising Gated Recurrent Units (2017, background): “Several works have already highlighted the effectiveness of RNNs in various speech processing tasks such as speech recognition [11, 12, 13, 14, 15], speech enhancement [16], speech separation [17, 18] as well as speech activity detection [19].”