Time-Frequency Masking in the Complex Domain for Speech Dereverberation and Denoising

By Donald S. Williamson, Deliang Wang. 2017

229 citations recorded by the source corpus.

In real-world situations, speech is masked by both background noise and reverberation, which negatively affect perceptual quality and intelligibility. In this paper, we address monaural speech separation in reverberant and noisy environments. We perform dereverberation and denoising using supervised learning with a deep neural network. Specifically, we enhance the magnitude and phase by performing separation with an estimate of the complex ideal ratio mask. We define the complex ideal ratio mask so that direct speech results after the mask is applied to reverberant and noisy speech. Our approach is evaluated using simulated and real room impulse responses, and with background noises. The proposed approach improves objective speech quality and intelligibility significantly. Evaluations and comparisons show that it outperforms related methods in many reverberant and noisy environments.

Subfields: Cognitive Neuroscience, Computational Mechanics, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computational Mechanics

In Speech & Audio

In Signal Processing

Citations in the corpus

SoundStream: An End-to-End Neural Audio Codec (2021, methodology): “Audio enhancement – Deep neural networks have been applied to different audio enhancement tasks, ranging from denoising [17]–[21] to dereverberation [22], [23], lossy coding denoising [2] and frequency bandwidth extension [1], [24].”
Speech Denoising with Deep Feature Losses (2018, background): “This issue can be alleviated somewhat, but with increased computational cost and system complexity [12, 13, 14, 15, 16, 17, 18].”
End-to-End Speech Separation with Unfolded Iterative Phase Reconstruction (2018, background): “[27, 28] proposed a complex ratio masking approach for phase reconstruction and speech enhancement, where a feed-forward DNN is trained to predict the real and imaginary components of the ideal complex filter in the STFT domain, i.”
Deep Learning Based Phase Reconstruction for Speaker Separation: A Trigonometric Perspective (2018, result): “This distinguishes our approach from studies that directly predict unbounded or unconstrained phase differences [22], [23], clean phases [24] and real and imaginary components of target sources [25], [26], or fully complex neural network approaches [27].”
Deep Learning for Environmentally Robust Speech Recognition (2017, background): “Both two phase-based masks were demonstrated to be more effective than normal IRMs in suppressing the reverberated noise in Williamson and Wang (2017b).”
Speech Dereverberation Using Fully Convolutional Networks (2018, methodology): “[14] used deep learning techniques to directly estimate the complex ideal ratio mask (cIRM).”
Speaker-Independent Speech Separation With Deep Attractor Network (2017, background): “ingle-class or multi-class regression problem. Different types of masks and objective functions have been proposed. For instance, phase-aware masks for enhancement and separation have been studied in [21], [29], [30]. Limitations of the previous neural networks become evident when one considers the ”
ReVISE: Self-Supervised Speech Resynthesis with Visual Input for Universal and Generalized Speech Enhancement (2022, methodology): “For these two subtasks, masking-based methods (in magnitude spectrogram domain [16, 41], complex spectrogram domain [14, 19, 55], or on learned basis [33, 34]) are widely adopted where the enhancement model f predicts a mask M̂ = f(x̃a, xv).”
Investigating Generative Adversarial Networks based Speech Dereverberation for Robust Speech Recognition (2018, background): “Either direct spectral mapping [4, 13] or masking [16] can be considered in the dereverberation network.”