A Convolutional Recurrent Neural Network for Real-Time Speech Enhancement

By Ke Tan, Deliang Wang. 2018

522 citations recorded by the source corpus.

Many real-world applications of speech enhancement, such as hearing aids and cochlear implants, desire real-time processing, with no or low latency. In this paper, we propose a novel convolutional recurrent network (CRN) to address real-time monaural speech enhancement. We incorporate a convolutional encoder-decoder (CED) and long short-term memory (LSTM) into the CRN architecture, which leads to a causal system that is naturally suitable for real-time processing. Moreover, the proposed model is noise-and speaker-independent, i.e. noise types and speakers can be different between training and test. Our experiments suggest that the CRN leads to consistently better objec-tive intelligibility and perceptual quality than an existing LSTM based model. Moreover, the CRN has much fewer trainable parameters.

Subfields: Computational Mechanics, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Signal Processing

Citations in the corpus

End-to-End Neural Speech Coding for Real-Time Communications (2022, positive): “…encoder-temporal ﬁltering-decoder paradigm based on causal convolutions commonly used for regression tasks in real-time audio signal processing [12, 13], in this paper we propose the TFNet, a low-latency neural speech codec with end-to-end optimization for real-time communications, following this…”
FRCRN: Boosting Feature Representation Using Frequency Recurrence for Monaural Speech Enhancement (2022, unknown): “The CRN has been shown to be very effective for speech enhancement [4, 5] and the extended complex-valued DCCRN [6] achieved the best performance in the Interspeech 2020 DNS Challenge.”
Learning Complex Spectral Mapping with Gated Convolutional Recurrent Networks for Monaural Speech Enhancement (2019, methodology): “Recently, we integrated a CED and LSTMs into a CRN, which amounts to a causal system [32].”
TCNN: Temporal Convolutional Neural Network for Real-time Speech Enhancement in the Time Domain (2019, unknown): “Second, we train another real-time causal system recently proposed in [19].”
DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancement (2020, background): “The convolution recurrent network (CRN), originally described in [14], is an essentially causal CED architecture with two LSTM layers between the encoder and the decoder.”
Data Augmentation and Loss Normalization for Deep Noise Suppression (2020, unknown): “in [6–10].”
Weighted Speech Distortion Losses for Neural-Network-Based Real-Time Speech Enhancement (2020, background): “It is worth mentioning that we do not apply convolution layers as often done in other related work [11, 25] because of the relatively arbitrary process involved in choosing the amount of frequency span and filter taps.”
Interactive Speech and Noise Modeling for Speech Enhancement (2020, background): “They typically estimate a real or complex ratio mask for each T-F bin to map noisy spectra to speech spectra (Williamson, Wang, and Wang 2015; Wang, Narayanan, and Wang 2014; Choi et al. 2019) or directly predict the speech representation (Xu et al. 2013; Tan and Wang 2018).”
Deep Learning Based Phase Reconstruction for Speaker Separation: A Trigonometric Perspective (2018, methodology): “To improve mask estimation, we employ a BLSTM with convolutional encoder-decoder structures and skip connections [14], which will be discussed in Section 4.”
MOSNet: Deep Learning based Objective Assessment for Voice Conversion (2019, background): “Recent studies have confirmed the effectiveness of combining CNN and RNN (BLSTM) for enhancement [17, 18], classification [19, 20], and recognition [21, 22] tasks.”