A Wavenet for Speech Denoising

By Dario Rethage, Jordi Pons, Xavier Serra. 2017

489 citations recorded by the source corpus.

Most speech processing techniques use magnitude spectrograms as front-end and are therefore by default discarding part of the signal: the phase. In order to overcome this limitation’ we propose an end-to-end learning method for speech denoising based on Wavenet. The proposed model adaptation retains Wavenet's powerful acoustic modeling capabilities, while significantly reducing its time-complexity by eliminating its autoregressive nature. Specifically, the model makes use of non-causal, dilated convolutions and predicts target fields instead of a single target sample. The discriminative adaptation of the model we propose, learns in a supervised fashion via minimizing a regression loss. These modifications make the model highly parallelizable during both training and inference. Both quantitative and qualitative evaluations indicate that the proposed method is preferred over Wiener filtering, a common method based on processing the magnitude spectrogram.

Subfields: Artificial Intelligence, Natural Language Processing, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Signal Processing

Citations in the corpus

DiffWave: A Versatile Diffusion Model for Audio Synthesis (2020, methodology): “3 The similar architecture has been applied for source separation (Rethage et al., 2018; Lluís et al., 2018).”
SEANet: A Multi-modal Speech Enhancement Network (2020, neutral): “Al-ternatively, generative models based on GANs [11, 12] and WaveNet [13] have been proposed.”
Speech Denoising with Deep Feature Losses (2018, methodology): “As baselines, we use a Wiener filtering pipeline with a priori noise SNR estimation (as implemented in [42]), and two recent state-of-the-art methods that use deep networks to perform endto-end denoising directly on the raw waveform: the Speech Enhancement Generative Adversarial Network (SEGAN) [23]”
Real Time Speech Enhancement in the Waveform Domain (2020, methodology): “Over the last years, there has been a growing interest towards DNN based methods for speech enhancement [34, 35, 36, 37, 20, 38, 39, 22, 40, 7, 41, 8, 42, 21, 37].”
Music Source Separation in the Waveform Domain (2019, methodology): “A Wavenet-inspired, although using a regression loss and not auto-regressive, was first used for speech denoising [Rethage et al., 2018] and then adapted to source separation [Lluís et al.”
DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism (2021, methodology): “We adopt a non-causal WaveNet [35] architecture proposed by [13, 28] as our denoiser8.”
Wave-U-Net: A Multi-Scale Neural Network for End-to-End Audio Source Separation (2018, methodology): “This type of output was also used for speech denoising in [18] as part of an “energy-conserving” loss, and a similar idea can be found very commonly in spectrogram-based source separation in the form of masks that distribute the energy of the input mixture magnitudes to the output sources.”
WG-WaveNet: Real-Time High-Fidelity Speech Synthesis without GPU (2020, methodology): “The output of WaveGlow is then used as the input of the WaveNet-based post-filter to generates x̂ in parallel [17, 18] conditioned on an upsampled Melspectrogram.”
End-to-end music source separation: is it possible in the waveform domain? (2018, background): “Provided that the GPU’s memory is limited, this experiment explores the trade-off between how many filters each Wavenet layer has (the GPU’s memory mostly stores learnable parameters with a wider Wavenet) and the receptive field length of the network (the GPU’s memory mostly stores feature maps with”
PoCoNet: Better Speech Enhancement with Frequency-Positional Embeddings, Semi-Supervised Conversational Data, and Biased Loss (2020, methodology): “Recent phase-aware models use a complex ratio mask instead of magnitude gain [6, 7], while other approaches work directly in the waveform domain [8, 9, 10].”
End-to-End Waveform Utterance Enhancement for Direct Evaluation Metrics Optimization by Fully Convolutional Neural Networks (2017, background): “In fact, other waveform enhancement frameworks based on generative adversarial networks (GANs) [48] and WaveNet [49], [50] were also shown to outperform”
Wavelet Networks: Scale Equivariant Learning From Raw Waveforms (2020, background): “[53] Dario Rethage, Jordi Pons, and Xavier Serra.”
Universal Speech Enhancement with Score-based Diffusion (2022, methodology): “With the advent of deep learning, the task has been dominated by neural networks, often outperforming more classical algorithms and generalizing to multiple noise types (Lu et al., 2013; Pascual et al., 2017; Rethage et al., 2018; Défossez et al., 2020; Fu et al., 2021).”
GlotNet—A Raw Waveform Model for the Glottal Excitation in Statistical Parametric Speech Synthesis (2019, methodology): “lutional networks have recently been applied to many tasks, including speech bandwidth extension [24] and denoising [25].”
Fftnet: A Real-Time Speaker-Dependent Neural Vocoder (2018, methodology): “The approach has also been adapted to other problems, including voice conversion [5, 6], speech enhancement [7], and musical instrument synthesis [1, 8].”
FullSubNet+: Channel Attention Fullsubnet with Complex Spectrograms for Speech Enhancement (2022, methodology): “The time-domain approaches [1–3] predict a clean speech waveform directly from the corresponding noisy speech waveform.”
A New Framework for CNN-Based Speech Enhancement in the Time Domain (2019, background): “The proposed model in [16] is very slow, and in [17], it operates at the frame level for efficiency and hence suffers from the same problem.”
High Fidelity Speech Regeneration with Application to Speech Enhancement (2021, methodology): “Deep generative models such as Generative Adversarial Network (GAN) or WaveNet were also suggested for the task [6, 7, 8].”
A Universal Music Translation Network (2018, methodology): “In [12], the authors have used it for denoising waveforms by predicting the middle ground-truth sample from”