Reverberant speech recognition combining deep neural networks and deep autoencoders augmented with a phone-class feature

By Masato Mimura, Shinsuke Sakai, Tatsuya Kawahara. 2015

34 citations recorded by the source corpus.

We propose an approach to reverberant speech recognition adopting deep learning in the front-end as well as b a c k-e n d o f a r e v e r b e r a n t s p e e c h r e c o g n i t i o n s y s t e m, a n d a n o v e l m e t h o d t o i m p r o v e t h e d e r e v e r b e r a t i o n p e r f o r m a n c e of the front-end network using phone-class information. At the front-end, we adopt a deep autoencoder (DAE) for enhancing the speech feature parameters, and speech recognition is performed in the back-end using DNN-HMM acoustic models trained on multi-condition data. The system was evaluated through the ASR task in the Reverb Challenge 2014. The DNN-HMM system trained on the multi-condition training set achieved a conspicuously higher word accuracy compared to the MLLR-adapted GMM-HMM system trained on the same data. Furthermore, feature enhancement with the deep autoencoder contributed to the improvement of recognition accuracy especially in the more adverse conditions. While the mapping between reverberant and clean speech in DAE-based dereverberation is conventionally conducted only with the acoustic information, we presume the mapping is also dependent on the phone information. Therefore, we propose a new scheme (pDAE), which augments a phone-class feature to the standard acoustic features as input. Two types of the phone-class feature are investigated. One is the hard recognition result of monophones, and the other is a soft representation derived from the posterior outputs of monophone DNN. The augmented feature in either type results in a significant improvement (7–8 % relative) from the standard DAE.

Subfields: Artificial Intelligence, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Speech & Audio

In Signal Processing

Citations in the corpus

The DIRHA-ENGLISH corpus and related tasks for distant-speech recognition in domestic environments (2015, background): “…at multi-microphone front-end processing level in order to feed ASR with an enhanced speech input [4, 5, 6, 7, 8], the performance loss observed from close-talking to distant-speech remains quite critical, even when the most advanced DNN-based backend frameworks are adopted [9, 10, 11, 12, 13, 14].”
Batch-normalized joint training for DNN-based distant speech recognition (2016, background): “Nevertheless, the recent success of deep learning has not only largely contributed to the substantial improvement of the speech recognition part of a DSR system [19, 20, 21, 22, 23, 24, 25], but has also enabled the de-a r X i v : 1703 .”
Contaminated speech training methods for robust DNN-HMM distant speech recognition (2017, background): “Concerning the latter field, the research community is currently experiencing a revolution due to the introduction of Deep Neural Networks (DNNs) [13], which have consistently outperformed previous Gaussian Mixture Models (GMMs) for both close [14] and distant-talking scenarios [15, 16, 17, 18, 19].”
A summary of the REVERB challenge: state-of-the-art and remaining challenges in reverberant speech processing research (2016, background): “More specifically, their processing strategies seem to commonly emphasize the joint utilization of the following: • front-end processing such as beamforming and dereverberation that effectively utilize multichannel acoustic diversity by linear filtering, • strong acoustic models such as DNNs, and • ”
Realistic Multi-Microphone Data Simulation for Distant Speech Recognition (2016, background): “Despite the recent substantial progress in various related fields, including spatial filtering [1, 2], microphone selection [3], source separation [4], speech dereverberation [5], speaker localization [6], acoustic event detection [7] as well as acoustic modeling [8, 9, 10, 11, 12], DSR still exhibi”