Personalized speech enhancement: new models and Comprehensive evaluation
By Sefik Emre Eskimez, Takuya Yoshioka, Huaming Wang, Xiaofei Wang, Zhuo Chen, Xuedong Huang. 2021
82 citations recorded by the source corpus.
Personalized speech enhancement (PSE) models utilize additional cues, such as speaker embeddings like d-vectors, to remove background noise and interfering speech in real-time and thus improve the speech quality of online video conferencing systems for various acoustic scenarios. In this work, we propose two neural networks for PSE that achieve superior performance to the previously proposed VoiceFilter. In addition, we create test sets that capture a variety of scenarios that users can encounter during video conferencing. Furthermore, we propose a new metric to measure the target speaker over-suppression (TSOS) problem, which was not sufficiently investigated before despite its critical importance in deployment. Besides, we propose multi-task training with a speech recognition back-end. Our results show that the proposed models can yield better speech recognition accuracy, speech intelligibility, and perceptual quality than the baseline models, and the multi-task training can alleviate the TSOS issue in addition to improving the speech recognition accuracy.
Subfields: Artificial Intelligence, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Ross Cutler
- Marc Delcroix
- Keisuke Kinoshita
- Vishak Gopal
- Tsubasa Ochiai
- Sebastian Braun
- Kateřina Žmolíková
- Chandan K. A. Reddy
In Signal Processing
- Ross Cutler
- Vishak Gopal
- Chandan K. A. Reddy
- Robert Aichner
- Hannes Gamper
- Sebastian Braun
- Harishchandra Dubey
- Marc Delcroix
Citations in the corpus
- SpeechX: Neural Codec Language Model as a Versatile Speech Transformer (2023, neutral): “Index Terms —Speech generation, audio-text input, multi-task learning, zero-shot text-to-speech, noise suppression, target speaker extraction, speech editing, speech removal.”
- Icassp 2022 Deep Noise Suppression Challenge (2022, methodology): “We use the personalized DCCRN (pDCCRN) model described in [19] as the baseline PDNS model for 4th DNS Challenge.”
- Neural Target Speech Extraction: An overview (2023, background): “In the literature, this is called personalized speech enhancement, which, in this article, we view as a special case of the TSE problem, where only the target speaker is actively speaking [15].”