A Fully Convolutional Neural Network for Speech Enhancement

By Se Rim Park, Jinwon Lee. Interspeech, 2016

401 citations recorded by the source corpus.

In hearing aids, the presence of babble noise degrades hearing intelligibility of human speech greatly. However, removing the babble without creating artifacts in human speech is a challenging task in a low SNR environment. Here, we sought to solve the problem by finding a `mapping' between noisy speech spectra and clean speech spectra via supervised learning. Specifically, we propose using fully Convolutional Neural Networks, which consist of lesser number of parameters than fully connected networks. The proposed network, Redundant Convolutional Encoder Decoder (R-CED), demonstrates that a convolutional network can be 12 times smaller than a recurrent network and yet achieves better performance, which shows its applicability for an embedded system: the hearing aids.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Signal Processing

View Paper: https://www.semanticscholar.org/paper/9ed8e2f6c338f4e0d1ab0d8e6ab8b836ea66ae95

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Signal Processing

Citations in the corpus

Deep Residual-Dense Lattice Network for Speech Enhancement (2020, background): “Convolutional neural networks (CNNs) were able to match the performance of LSTM networks, with fewer parameters and a reduction in training time (Park and Lee 2017).”
Investigating Generative Adversarial Networks based Speech Dereverberation for Robust Speech Recognition (2018, background): “The structure of the RCED is similar with [15] except the last layer.”
Joint Phoneme Alignment and Text-Informed Speech Separation on Highly Corrupted Speech (2020, methodology): “Several deep learning approaches such as Long Short-Term Memory (LSTM) cells [4, 10], convolutional neural networks [11], and generative adversarial networks [12] have been applied to speech separation.”
Gated Residual Networks with Dilated Convolutions for Supervised Speech Separation (2018, neutral): “Previous CNNs [7] [8] [9] for speech separation yield relatively small improvements compared to DNNs, while the LSTM model significantly outperforms a DNN model with a large number of training speakers [6].”
PoCoNet: Better Speech Enhancement with Frequency-Positional Embeddings, Semi-Supervised Conversational Data, and Biased Loss (2020, methodology): “On the other hand, in standard 2D U-Net models where kernels move in both the time and frequency directions [13], early layer activations are blind to what frequency they operate in – even in the case when padding is used, these early features’ receptive fields have not yet reached the edges of the ”
Speech Enhancement of Noisy and Reverberant Speech for Text-to-Speech (2018, methodology): “based systems [52], [53] and convolutional neural networks [54], [55].”
Exploiting time-frequency patterns with LSTM-RNNs for low-bitrate audio restoration (2019, methodology): “Owing to these successes, CNNs have been increasingly employed for acoustic analysis as well, such as speech recognition [3], speaker analysis [36], music information retrieval [25], and denoising [44].”
Learning Complex Spectral Mapping with Gated Convolutional Recurrent Networks for Monaural Speech Enhancement (2019, methodology): “In addition, CNNs have also been used for mask estimation and spectral mapping [7], [25], [11], [31].”
Deep Learning for Environmentally Robust Speech Recognition (2017, neutral): “[36], [63], [77], [99] invertible to the audio signal high dimension, require high computational load, each element is equally important temporal [79], [80] retain the complete information large data size, require heavy computational load”
Speech Enhancement for Noise-Robust Speech Synthesis Using Wasserstein GAN (2019, background): “Different types of networks have also been investigated in the literature for enhancement, such as autoencoders [5], convolution networks [6] and recurrent networks [7].”
Conditional Generative Adversarial Networks for Speech Enhancement and Noise-Robust Speaker Verification (2017, methodology): “Lately, deep learning methods have been used, such as DNNs [6, 8], deep autoencoders (DAEs) [5], and convolutional neural networks (CNNs) [9].”
FRCRN: Boosting Feature Representation Using Frequency Recurrence for Monaural Speech Enhancement (2022, neutral): “Many deep models of different structures have been studied for speech enhancement, such as feedforward neural networks (FNN) [1], recurrent neural networks (RNN) [2] and convolutional neural networks (CNN) [3].”
Speech Enhancement with a Generative Adversarial Network (2019, methodology): “A range of deep neural nets have later been applied: convolutional neural nets (CNNs) (Hui et al., 2015; Fu et al., 2016; Park and Lee, 2016), recurrent neural nets (RNNs) (Weninger et al.”
A Convolutional Recurrent Neural Network for Real-Time Speech Enhancement (2018, neutral): “[17] first introduced CED for speech enhancement.”
Raw waveform-based speech enhancement by fully convolutional networks (2017, background): “for SE [30] to process the magnitude spectrum.”
Convolutional Neural Networks to Enhance Coded Speech (2018, methodology): “It is also because of this property that the number of the trainable weights is reduced in a large scale compared to DNNs and RNNs with fully-connected structures, making it more efficient to train the network [32].”
A New Framework for CNN-Based Speech Enhancement in the Time Domain (2019, neutral): “Among the most popular deep learning methods are denoising autoencoders [4], feedforward neural networks [5], [6], and CNNs [7]–[9].”
Supervised Speech Separation Based on Deep Learning: An Overview (2017, neutral): “CNNs have also been used for IRM estimation [83] and 773 spectral mapping [46], [136], [138].”