Conditional Diffusion Probabilistic Model for Speech Enhancement
By Yen-Ju Lu, Zhongqiu Wang, Shinji Watanabe, Alexander Richard, Cheng Yu, Yu Tsao. 2022
299 citations recorded by the source corpus.
Speech enhancement is a critical component of many user-oriented audio applications, yet current systems still suffer from distorted and unnatural outputs. While generative models have shown strong potential in speech synthesis, they are still lagging behind in speech enhancement. This work leverages recent advances in diffusion probabilistic models, and proposes a novel speech enhancement algorithm that incorporates characteristics of the observed noisy speech signal into the diffusion and reverse processes. More specifically, we propose a generalized formulation of the diffusion probabilistic model named conditional diffusion probabilistic model that, in its reverse process, can adapt to non-Gaussian real noises in the estimated speech signal. In our experiments, we demonstrate strong performance of the proposed approach compared to representative generative models, and investigate the generalization capability of our models to other datasets with noise characteristics unseen during training.
Subfields: Artificial Intelligence, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Navonil Majumder
- Soujanya Poria
- Deepanway Ghosal
- Jordi Pons
- Santiago Pascual
- Joan Serrà
- Dongchao Yang
- Recep Oguz Araz
In Signal Processing
- Jordi Pons
- Santiago Pascual
- Joan Serrà
- Recep Oguz Araz
- Davide Scaini
- Navonil Majumder
- Dongchao Yang
- Soujanya Poria
Citations in the corpus
- NADiffuSE: Noise-aware Diffusion-based Model for Speech Enhancement (2023, methodology): “The mean is a linear combination of x t , y and the estimated noise term ϵ , where c xt , c yt and c ϵt are constants and the detailed derivation follows [23].”
- A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AI (2023, unknown): “In contrast to DiffuSE [62] and CDiffuSE [63] performing diffusion on the time-domain waveform, SGMSE [124] investigates speech enhancement on the time-frequency short-”
- Speech Enhancement and Dereverberation With Diffusion-Based Generative Models (2022, unknown): “Thus, current diffusion-based models for speech enhancement, such as [21]–[23], can be considered conditional generative models trained in a supervised manner.”
- Universal Speech Enhancement with Score-based Diffusion (2022, background): “(Phan et al., 2020) 2.39 2.90 0.93 SE-Flow (Strauss & Edler, 2021) 2.43 3.09 DiffuSE (Lu et al., 2021) 2.44 3.03 CDiffuSE (Lu et al., 2022) 2.52 3.10 PR-WaveGlow (Maiti & Mandel, 2020) 3.10 0.91 CycleGAN-DCD (Yu et al., 2021) 2.90 3.49 0.94 PSMGAN (Routray & Mao, 2022) 2.92 3.52 PoCoNet (Isik et al.”
- Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Model (2023, background): “Diffusion models have emerged as a promising approach for speech processing, particularly in speech enhancement [21, 37, 29, 22].”