TCNN: Temporal Convolutional Neural Network for Real-time Speech Enhancement in the Time Domain

By Ashutosh Pandey, Deliang Wang. 2019

382 citations recorded by the source corpus.

This work proposes a fully convolutional neural network (CNN) for real-time speech enhancement in the time domain. The proposed CNN is an encoder-decoder based architecture with an additional temporal convolutional module (TCM) inserted between the encoder and the decoder. We call this architecture a Temporal Convolutional Neural Network (TCNN). The encoder in the TCNN creates a low dimensional representation of a noisy input frame. The TCM uses causal and dilated convolutional layers to utilize the encoder output of the current and previous frames. The decoder uses the TCM output to reconstruct the enhanced frame. The proposed model is trained in a speaker- and noise-independent way. Experimental results demonstrate that the proposed model gives consistently better enhancement results than a state-of-the-art real-time convolutional recurrent model. Moreover, since the model is fully convolutional, it has much fewer trainable parameters than earlier models.

Subfields: Computational Mechanics, Electrical and Electronic Engineering, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. Yuzhou Liu
2. Yan Lu
3. Xiulian Peng
4. Chengyu Zheng
5. Yuan Zhang
6. Huaying Xue
7. Xue Jiang
8. Radu Patrice Horaud

In Electrical and Electronic Engineering

1. Yan Lu
2. Xiulian Peng
3. Yuzhou Liu
4. Huaying Xue
5. Xue Jiang
6. Chengyu Zheng
7. Yuan Zhang
8. Radu Patrice Horaud

In Signal Processing

1. Xiulian Peng
2. Yan Lu
3. Xue Jiang
4. Yuzhou Liu
5. Zhiyong Wu
6. Yuan Zhang
7. Chengyu Zheng
8. Radu Patrice Horaud

Citations in the corpus

• End-to-End Neural Speech Coding for Real-Time Communications (2022, positive): “…the encoder-temporal filtering-decoder paradigm based on causal convolutions commonly used for regression tasks in real-time audio signal processing [12, 13], in this paper we propose the TFNet, a low-latency neural speech codec with end-to-end optimization for real-time communications, following…”
• Interactive Speech and Noise Modeling for Speech Enhancement (2020, methodology): “Some methods operate in the time domain by estimating speech signals from raw-waveform noisy signals in an end-to-end way (Fu et al. 2017; Pascual, Bonafonte, and Serra 2017; Pandey and Wang 2019).”
• Cross-Scale Vector Quantization for Scalable Neural Speech Coding (2022, positive): “D convolutional layers, followed by a temporal convolution module (TCM) similar to that in [21] and a GRU layer for long-term temporal correlation exploitation.”
• Divide and Conquer: A Deep CASA Approach to Talker-Independent Monaural Speaker Separation (2019, methodology): “al grouping stage, we introduce a new embedding representation and weighted objective function. In addition, we leverage the latest development in temporal convolutional networks (TCNs) [2] [22] [27] [29], and use a TCN for sequential grouping, which greatly improves speaker tracking. A new dropout ”
• FullSubNet+: Channel Attention Fullsubnet with Complex Spectrograms for Speech Enhancement (2022, methodology): “The time-domain approaches [1–3] predict a clean speech waveform directly from the corresponding noisy speech waveform.”
• Fullsubnet: A Full-Band and Sub-Band Fusion Model for Real-Time Single-Channel Speech Enhancement (2020, methodology): “The time-domain methods [1–3] use the neural network to map noisy speech waveform to clean speech waveform directly.”