Improving wideband speech recognition using mixed-bandwidth training data in CD-DNN-HMM

By Jinyu Li, Dong Yu, Jui-Ting Huang, Yifan Gong. 2012

147 citations recorded by the source corpus.

Context-dependent deep neural network hidden Markov model (CD-DNN-HMM) is a recently proposed acoustic model that significantly outperformed Gaussian mixture model (GMM)-HMM systems in many large vocabulary speech recognition (LVSR) tasks. In this paper we present our strategy of using mixed-bandwidth training data to improve wideband speech recognition accuracy in the CD-DNN-HMM framework. We show that DNNs provide the flexibility of using arbitrary features. By using the Mel-scale log-filter bank features we not only achieve higher recognition accuracy than using MFCCs, but also can formulate the mixed-bandwidth training problem as a missing feature problem, in which several feature dimensions have no value when narrowband speech is presented. This treatment makes training CD-DNN-HMMs with mixed-bandwidth data an easy task since no bandwidth extension is needed. Our experiments on voice search data indicate that the proposed solution not only provides higher recognition accuracy for the wideband speech but also allows the same CD-DNN-HMM to recognize mixed-bandwidth speech. By exploiting mixed-bandwidth training data CD-DNN-HMM outperforms fMPE+BMMI trained GMM-HMM, which cannot benefit from using narrowband data, by 18.4%.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Deep Learning: Methods and Applications (2014, methodology): “A DNN architecture for speech recognition trained with mixed-bandwidth acoustic data with 16-kHz and 8-kHz sampling rates; [after (Li et al., 2012), @IEEE].”
Feature Learning in Deep Neural Networks - A Study on Speech Recognition Tasks (2013, methodology): “One such system was recently proposed using a CD-DNN-HMM [17].”
Extracting Domain Invariant Features by Unsupervised Learning for Robust Automatic Speech Recognition (2018, background): “Another popular strategy is to train an ASR system with as large, and as diverse a dataset as possible [8, 9]; however, this strategy is not feasible when the labeled data are not available for all domains.”
A Better and Faster end-to-end Model for Streaming ASR (2020, unknown): “In addition to the diverse training sets, multi-condition training (MTR) [37] and random data down-sampling to 8kHz [38] are also used to further increase data diversity.”
An Overview of Noise-Robust Automatic Speech Recognition (2014, methodology): “With deep and wide hidden layers, the DNN provides a very strong normalization to heterogeneous data [77], [78], [287].”
Joint Optimization of Masks and Deep Recurrent Neural Networks for Monaural Source Separation (2015, methodology): “In the speech recognition literature [37], the log-mel filterbank is found to provide lower word-error-rate compared to mel-frequency cepstral coefficients (MFCC) and log FFT bins.”
A Streaming On-Device End-To-End Model Surpassing Server-Side Conventional Model Quality and Latency (2020, methodology): “2, multi-condition training (MTR) [27, 28] and random data downsampling to 8kHz [29] are also used to further increase data diversity.”
Unsupervised adaptation with domain separation networks for robust speech recognition (2017, methodology): “The 29-dimensional log Mel filterbank features together with 1st and 2nd order delta features (totally 87-dimensional) for both the clean and noisy utterances are extracted by following the process in [31].”
Conditional Teacher-student Learning (2019, methodology): “The 29-dimensional log Mel filterbank features together with 1st and 2nd order delta features (totally 87-dimensional) for both the clean and noisy utterances are extracted by following the process in [40].”
Recent progresses in deep learning based acoustic models (2017, methodology): “Typically, log Mel-ﬁlter-bank features are often used as the input to the neural-network-based acoustic model [33], [34].”
Feature dimensionality reduction: a review (2022, methodology): “People have conducted deep research on DNN (Deep Neural Network) instead of GMM (Gaussian Mixture Model) to compose the DNN-HMM acoustic model [169, 170].”
Speech emotion recognition using deep neural network and extreme learning machine (2014, unknown): “In addition, motivated by the recent progress in speech recognition [16, 17], we have attempted to train the DNN directly using the filterbank or spectral features, but the performance is not satisfactory.”
Speaker-Invariant Training Via Adversarial Learning (2018, methodology): “The 29-dimensional log Mel ﬁlterbank features together with 1st and 2nd order delta features (totally 87-dimensional) for both the clean and noisy utterances are extracted by following the process in [29].”
Less is More: Improved RNN-T Decoding Using Limited Label Context and Path Merging (2020, methodology): “We employ mixed-bandwidth training [31], by randomly downsampling the training data to 8KHz 50% of the time.”
Convolutional Neural Networks for Distant Speech Recognition (2014, methodology): “The complete set of convolutional layer activations h = [h1, . . . ,hK ] 2 RK⇥J is composed of K = (B F )/L + 1 convolutional bands obtained by applying the (shared) set of J filters across the whole (multi-channel) input space V (as depicted in Fig 1).”
Elastic spectral distortion for low resource speech recognition with deep neural networks (2013, result): “On the other hand, one previous study [4] showed that DNNs had a severely degraded performance when the acoustic properties of test samples were very different from those of the training samples.”