A New Framework for CNN-Based Speech Enhancement in the Time Domain

By Ashutosh Pandey, Deliang Wang. 2019

261 citations recorded by the source corpus.

This paper proposes a new learning mechanism for a fully convolutional neural network (CNN) to address speech enhancement in the time domain. The CNN takes as input the time frames of noisy utterance and outputs the time frames of the enhanced utterance. At the training time, we add an extra operation that converts the time domain to the frequency domain. This conversion corresponds to simple matrix multiplication, and is hence differentiable implying that a frequency domain loss can be used for training in the time domain. We use mean absolute error loss between the enhanced short-time Fourier transform (STFT) magnitude and the clean STFT magnitude to train the CNN. This way, the model can exploit the domain knowledge of converting a signal to the frequency domain for analysis. Moreover, this approach avoids the well-known invalid STFT problem since the proposed CNN operates in the time domain. Experimental results demonstrate that the proposed method substantially outperforms the other methods of speech enhancement. The proposed method is easy to implement and applicable to related speech processing tasks that require time-frequency masking or spectral mapping.

Subfields: Artificial Intelligence, Computational Mechanics, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yuzhou Liu
2. Deliang Wang
3. Arun Narayanan
4. Yuxuan Wang
5. Bo Li
6. Yi Luo
7. Khe Chai Sim
8. Zhuo Chen

In Computational Mechanics

1. Yuzhou Liu
2. Deliang Wang
3. Yuxuan Wang
4. Ke Tan
5. Donald S. Williamson
6. Arun Narayanan
7. Zhuo Chen
8. Kun Han

In Signal Processing

1. Yuzhou Liu
2. Deliang Wang
3. Yuxuan Wang
4. Arun Narayanan
5. Yi Luo
6. Ke Tan
7. Zhuo Chen
8. Bo Li

Citations in the corpus

• A Spectral Energy Distance for Parallel Speech Synthesis (2020, neutral): “For example, the probability distillation methods ClariNet [27] and Parallel WaveNet [36] minimize the distance between spectrogram magnitudes of real and synthesized speech in addition to their main distillation loss; and Pandey and Wang [26] use a spectrogram-based loss for speech enhancement.”
• Divide and Conquer: A Deep CASA Approach to Talker-Independent Monaural Speaker Separation (2019, methodology): “performance in various tasks [2], [23], [29], [31].”