Weighted Speech Distortion Losses for Neural-Network-Based Real-Time Speech Enhancement

By Yangyang Raymond Xia, Sebastian Braun, Chandan K. A. Reddy, Harishchandra Dubey, Ross Cutler, Ivan J. Tashev. 2020

137 citations recorded by the source corpus.

This paper investigates several aspects of training a RNN (recurrent neural network) that impact the objective and subjective quality of enhanced speech for real-time single-channel speech enhancement. Specifically, we focus on a RNN that enhances short-time speech spectra on a single-frame-in, single-frame-out basis, a framework adopted by most classical signal processing methods. We propose two novel mean-squared-error-based learning objectives that enable separate control over the importance of speech distortion versus noise reduction. The proposed loss functions are evaluated by widely accepted objective quality and intelligibility measures and compared to other competitive online methods. In addition, we study the impact of feature normalization and varying batch sequence lengths on the objective quality of enhanced speech. Finally, we show subjective ratings for the proposed approach and a state-of-the-art real-time RNN-based method.

Subfields: Artificial Intelligence, Computational Mechanics, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Mechanics

In Signal Processing

Citations in the corpus

Real Time Speech Enhancement in the Waveform Domain (2020, background): “…causal methods, the authors in [46] pro-pose a convolutional recurrent network at the spectral level for real-time speech enhancement, while Xia, Yangyang, et al. [30] suggest to remove the convolutional layers and apply a weighted loss function to further improve results in the real-time setup.”
Sequence to Multi-Sequence Learning via Conditional Chain Mapping for Mixture Signals (2020, result): “Here, we compare the results with the official baseline model [47] and our implemented baseline based on TasNet.”
ICASSP 2022 Acoustic Echo Cancellation Challenge (2022, methodology): “We adapt a noise suppression model developed in [19] to the task of echo cancellation.”
The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Testing Framework, and Challenge Results (2020, methodology): “As a baseline, we will use the recently developed SE method from [16] which is based on Recurrent Neural Network (RNN).”
Fullsubnet: A Full-Band and Sub-Band Fusion Model for Real-Time Single-Channel Speech Enhancement (2020, methodology): “In addition to these two models, we also compared with the top-ranked methods in the DNS challenge (INTERSPEECH 2020), including NSNet [22], DTLN [23], Conv-TasNet [24], DCCRN [19] and PoCoNet [25].”
FB-MSTCN: A Full-Band Single-Channel Speech Enhancement Method Based on Multi-Scale Temporal Convolutional Network (2022, methodology): “In Table 2, we compare the proposed FB-MSTCN with the baseline model NSNet [20], and present the evaluated results of subjective speech quality (SIG), background noise quality (BAK), overall audio quality (OVRL) with ITU-T P.”
Data Augmentation and Loss Normalization for Deep Noise Suppression (2020, unknown): “in [6–10].”
Interactive Speech and Noise Modeling for Speech Enhancement (2020, background): “There are also some attempts on incorporating noise information, for example, by adding constraints to loss functions (Fan et al. 2019; Xu, Elshamy, and Fingscheidt 2020; Xia et al. 2020) or by directly predicting noise instead of speech (Odelowo and Anderson 2017, 2018).”