Improving Robustness of Deep Neural Network Acoustic Models via Speech Separation and Joint Adaptive Training

By Arun Narayanan, Deliang Wang. 2015

77 citations recorded by the source corpus.

Although deep neural network (DNN) acoustic models are known to be inherently noise robust, especially with matched training and testing data, the use of speech separation as a frontend and for deriving alternative feature representations has been shown to improve performance in challenging environments. We first present a supervised speech separation system that significantly improves automatic speech recognition (ASR) performance in realistic noise conditions. The system performs separation via ratio time-frequency masking; the ideal ratio mask (IRM) is estimated using DNNs. We then propose a framework that unifies separation and acoustic modeling via joint adaptive training. Since the modules for acoustic modeling and speech separation are implemented using DNNs, unification is done by introducing additional hidden layers with fixed weights and appropriate network architecture. On the CHiME-2 medium-large vocabulary ASR task, and with log mel spectral features as input to the acoustic model, an independently trained ratio masking frontend improves word error rates by 10.9% (relative) compared to the noisy baseline. In comparison, the jointly trained system improves performance by 14.4%. We also experiment with alternative feature representations to augment the standard log mel features, like the noise and speech estimates obtained from the separation module, and the standard feature set used for IRM estimation. Our best system obtains a word error rate of 15.4% (absolute), an improvement of 4.6 percentage points over the next best result on this corpus.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Exploring Speech Enhancement with Generative Adversarial Networks for Robust Speech Recognition (2017, methodology): “While deep learning has previously been applied to SFM for ASR [17, 18, 19], our work is the first to use GANs for this task.”
Single Channel Target Speaker Extraction and Recognition with Speaker Beam (2018, methodology): “Solving such mismatch between a speech enhancement front-end and an ASR back-end has been addressed by jointly training both modules [15, 21, 22].”
Deep Learning for Environmentally Robust Speech Recognition (2017, unknown): “22.20% [72] mapping: log Mel re-training BLSTM 22.16% [94] masking: PSM (SA; log Mel) BLSTM 14.76 (SDR) [63] mapping: log mag hybrid DNN ≈25% [110] masking: IRM (SA; log Mel) MTL; hybrid BLSTM 16.04% [144] masking: IRM (MA; Mel) joint DNN 15.40% [60] masking: IRM (PSA; Mel) BLSTM 13.76% [114] maskin”
Speech Enhancement with LSTM Recurrent Neural Networks and its Application to Noise-Robust ASR (2015, methodology): “Supervised training of speech enhancement schemes is becoming increasingly popular especially in the context of single-channel speech enhancement in nonstationary noise [16, 7].”