PoCoNet: Better Speech Enhancement with Frequency-Positional Embeddings, Semi-Supervised Conversational Data, and Biased Loss

By Umut Isik, Ritwik Giri, Neerad Phansalkar, Jean-Marc Valin, Karim Helwani, A. Krishnaswamy. 2020

91 citations recorded by the source corpus.

Neural network applications generally benefit from larger-sized models, but for current speech enhancement models, larger scale networks often suffer from decreased robustness to the variety of real-world use cases beyond what is encountered in training data. We introduce several innovations that lead to better large neural networks for speech enhancement. The novel PoCoNet architecture is a convolutional neural network that, with the use of frequency-positional embeddings, is able to more efficiently build frequency-dependent features in the early layers. A semi-supervised method helps increase the amount of conversational training data by pre-enhancing noisy datasets, improving performance on real recordings. A new loss function biased towards preserving speech quality helps the optimization better match human perceptual opinions on speech quality. Ablation experiments and objective and human opinion metrics show the benefits of the proposed improvements.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Human Listening and Live Captioning: Multi-Task Training for Speech Enhancement (2021, background): “In recent years, deep neural networks have shown great potential for single-channel speech enhancement (SE) (or noisesuppression) [2, 3, 4, 5, 6, 7, 8].”
Fullsubnet: A Full-Band and Sub-Band Fusion Model for Real-Time Single-Channel Speech Enhancement (2020, methodology): “PoCoNet [25] is a convolutional neural network with frequency-positional embeddings employed.”
Universal Speech Enhancement with Score-based Diffusion (2022, background): “…2.52 3.10 PR-WaveGlow (Maiti & Mandel, 2020) 3.10 0.91 CycleGAN-DCD (Yu et al., 2021) 2.90 3.49 0.94 PSMGAN (Routray & Mao, 2022) 2.92 3.52 PoCoNet (Isik et al., 2020) 2.75 3.42 FullSubNet (Hao et al., 2021) 3.31 2.78 0.96 DCCRN+ (Lv et al., 2021) 2.84 3.33 CTS-Net (Li et al., 2021a) 2.92 3.59…”
ICASSP 2021 Deep Noise Suppression Challenge (2020, methodology): “4 Differential Mean Opinion Score (DMOS) from the ideal Mean Opinion Score (MOS) of 5 when tested on the DNS Challenge test set [6, 7].”
Interactive Speech and Noise Modeling for Speech Enhancement (2020, methodology): “2020) and PoCoNet (Isik et al. 2020) are non-real-time methods, among which the PoCoNet took 1st place in the 2020 DNS challenge’s Non-Real-Time track.”