PHASEN: A Phase-and-Harmonics-Aware Speech Enhancement Network

By Dacheng Yin, Chong Luo, Zhiwei Xiong, Wenjun Zeng. AAAI Conference on Artificial Intelligence, 2019

402 citations recorded by the source corpus.

Time-frequency (T-F) domain masking is a mainstream approach for single-channel speech enhancement. Recently, focuses have been put to phase prediction in addition to amplitude prediction. In this paper, we propose a phase-and-harmonics-aware deep neural network (DNN), named PHASEN, for this task. Unlike previous methods which directly use a complex ideal ratio mask to supervise the DNN learning, we design a two-stream network, where amplitude stream and phase stream are dedicated to amplitude and phase prediction. We discover that the two streams should communicate with each other, and this is crucial to phase prediction. In addition, we propose frequency transformation blocks to catch long-range correlations along the frequency axis. Visualization shows that the learned transformation matrix implicitly captures the harmonic correlation, which has been proven to be helpful for T-F spectrogram reconstruction. With these two innovations, PHASEN acquires the ability to handle detailed phase patterns and to utilize harmonic patterns, getting 1.76dB SDR improvement on AVSpeech + AudioSet dataset. It also achieves significant gains over Google's network on this dataset. On Voice Bank + DEMAND dataset, PHASEN outperforms previous methods by a large margin on four metrics.

Subfields: Cognitive Neuroscience, Computational Mechanics, Signal Processing

View Paper: https://www.semanticscholar.org/paper/09b1ed9ff362ebf9a1be376ec8bfb59e2cc8c2c9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computational Mechanics

In Signal Processing

Citations in the corpus

Human Listening and Live Captioning: Multi-Task Training for Speech Enhancement (2021, methodology): “For the loss function of the SE-step and the SE model pretraining, we use the PHASEN loss function [4].”
Phase-aware Single-stage Speech Denoising and Dereverberation with U-Net (2020, methodology): “Another line of research is to sequentially estimate the clean phase part using an additional sub-module [13, 14, 15].”
Personalized speech enhancement: new models and Comprehensive evaluation (2021, background): “Deep neural networks (DNNs) have shown promising results compared with the traditional signal processing methods for single-channel speech enhancement (SE) [2, 3, 4, 5].”
Denoispeech: Denoising Text to Speech with Frame-Level Noise Modeling (2020, methodology): “…noisy corpus or the ground-truth noisy recordings of real-world noisy corpus; 3) Augment-Adversarial , apply data augmentation and adversarial factorization to FastSpeech 2 as used in [7] ; 4) Enhancement-Based , FastSpeech 2 trained on enhanced speech denoised with an enhancement model [5].”
Exploring the Best Loss Function for DNN-Based Low-latency Speech Enhancement with Temporal Convolutional Networks (2020, methodology): “From the table, it can be seen that STFT-TCN performs slightly worse than PHASEN [10].”
Interactive Speech and Noise Modeling for Speech Enhancement (2020, methodology): “PHASEN (Yin et al. 2020) is a two-branch T-F domain approach where one branch predicts the amplitude and the other predicts the phase.”
FullSubNet+: Channel Attention Fullsubnet with Complex Spectrograms for Speech Enhancement (2022, background): “Overall, since the frequency domain signal has more distinguishable features for model learning [11], the frequency-domain approaches dominate the most SOTA speech enhancement methods.”
FRCRN: Boosting Feature Representation Using Frequency Recurrence for Monaural Speech Enhancement (2022, neutral): “We choose a CRN-based DCCRN [6] model and a two-stream network, PHASEN [15], as baselines.”
Harmonic Gated Compensation Network Plus for ICASSP 2022 DNS Challenge (2022, background): “[8] verifies the necessity of modeling the harmonic.”