The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Testing Framework, and Challenge Results

By Chandan K. A. Reddy, Vishak Gopal, Ross Cutler, Ebrahim Beyrami, R. Cheng, Harishchandra Dubey, Sergiy Matusevych, Robert Aichner, Ashkan Aazami, Sebastian Braun and 3 more. 2020

460 citations recorded by the source corpus.

The INTERSPEECH 2020 Deep Noise Suppression (DNS) Challenge is intended to promote collaborative research in real-time single-channel Speech Enhancement aimed to maximize the subjective (perceptual) quality of the enhanced speech. A typical approach to evaluate the noise suppression methods is to use objective metrics on the test set obtained by splitting the original dataset. While the performance is good on the synthetic test set, often the model performance degrades significantly on real recordings. Also, most of the conventional objective metrics do not correlate well with subjective tests and lab subjective tests are not scalable for a large test set. In this challenge, we open-sourced a large clean speech and noise corpus for training the noise suppression models and a representative test set to real-world scenarios consisting of both synthetic and real recordings. We also open-sourced an online subjective test framework based on ITU-T P.808 for researchers to reliably test their developments. We evaluated the results using P.808 on a blind test set. The results and the key learnings from the challenge are discussed. The datasets and scripts can be found here for quick access this https URL.

Subfields: Biomedical Engineering, Cognitive Neuroscience, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Cognitive Neuroscience

In Signal Processing

Citations in the corpus

Icassp 2022 Deep Noise Suppression Challenge (2022, background): “An increasing number of recent DNS research papers are leveraging DNS Challenge datasets for training and validating their models [7, 4].”
Dnsmos: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressors (2020, methodology): “More details about the creation of these test sets can be found in [11].”
Torchaudio-Squim: Reference-Less Speech Quality and Intelligibility Measures in Torchaudio (2023, methodology): “We use the DNS Challenge 2020 [29] dataset in our experiments.”
Interactive Speech and Noise Modeling for Speech Enhancement (2020, unknown): “DNS Challenge The DNS challenge (Reddy et al. 2020) at Interspeech 2020 provides a large dataset for training.”
DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancement (2020, methodology): “Table 3: MOS on DNS challenge blind test set [1]”
ICASSP 2022 Acoustic Echo Cancellation Challenge (2022, methodology): “The second dataset is synthesized from speech recordings, room impulse responses, and background noise derived from [9].”
SpeechBrain: A General-Purpose Speech Toolkit (2021, methodology): “MetricGAN+ [39] Mimic Loss [40] VoiceBank [41] DNS [42] Speech separation Separate overlapped speech.”
Generative Pre-training for Speech with Flow Matching (2023, methodology): “In addition, we also trained our model using 100 hours of noisy speech from Deep Noise Supression Challenge 2020 (DNS2020; Reddy et al., 2020) for extra results to demonstrate the generalizability for SpeechFlow.”
PoCoNet: Better Speech Enhancement with Frequency-Positional Embeddings, Semi-Supervised Conversational Data, and Biased Loss (2020, methodology): “The proposed model took 1st place in the 2020 Deep Noise Suppression Challenge’s Non-Real-Time Track [16].”
Subjective Evaluation of Noise Suppression Algorithms in Crowdsourcing (2021, methodology): “We used the blind test set from the Deep Noise Suppression (DNS) Challenge [22] and applied 5 DNS models on that test set.”
NISQA: A Deep CNN-Self-Attention Model for Multidimensional Speech Quality Prediction with Crowdsourced Datasets (2021, background): “To simulate real background noises, the noise clips from the DNS-Challenge datasets were used, which in turn are taken from the Audioset [28], freesound [29], and DE-MAND [30] corpora.”
FRCRN: Boosting Feature Representation Using Frequency Recurrence for Monaural Speech Enhancement (2022, unknown): “We further conducted evaluations on two popular benchmarks: DNS-2020 dataset [11] and VoiceBank+Demand dataset [12] 2 and 3 show the performance comparisons with the published results of the SOTA models.”
Universal Speech Enhancement with Score-based Diffusion (2022, methodology): “Two widely-used data sets with an established test partition are Voicebank-DEMAND (Valentini-Botinhao, 2017) and IS20 DNS-Challenge (Reddy et al., 2020).”
Speech Enhancement and Dereverberation With Diffusion-Based Generative Models (2022, methodology): “For the evaluation, we use 300 files from the test set of the Deep Noise Suppression (DNS) Challenge 2020 [85].”