Dnsmos P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressors

By Chandan K. A. Reddy, Vishak Gopal, Ross Cutler. 2021

419 citations recorded by the source corpus.

Human subjective evaluation is the "gold standard" to evaluate speech quality optimized for human perception. Perceptual objective metrics serve as a proxy for subjective scores. We have recently developed a non-intrusive speech quality metric called Deep Noise Suppression Mean Opinion Score (DNSMOS) using the scores from ITU-T Rec. P.808 [1] subjective evaluation. The P.808 scores reflect the overall quality of the audio clip. ITU-T Rec. P.835 [2] subjective evaluation framework gives the standalone quality scores of speech and background noise in addition to the overall quality. In this work, we train an objective metric based on P.835 human ratings that output 3 scores: i) speech quality (SIG), ii) background noise quality (BAK), and iii) the overall quality (OVRL) of the audio. The developed metric is highly correlated with human ratings, with a Pearson’s Correlation Co-efficient (PCC)=0.94 for SIG and PCC=0.98 for BAK and OVRL. This is the first non-intrusive P.835 predictor we are aware of. DNSMOS P.835 is made publicly available as an Azure service.

Subfields: Cognitive Neuroscience, Computational Mechanics, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computational Mechanics

In Signal Processing

Citations in the corpus

Audiodec: An Open-Source Streaming High-Fidelity Neural Audio Codec (2023, methodology): “To evaluate the speech quality, mel-ceptral distortion (MCD), logspectral distortion (LSD), and non-intrusive speech quality metric Deep Noise Suppression Mean Opinion Score (DNSMOS) [37] were adopted.”
ICASSP 2023 Deep Noise Suppression Challenge (2023, methodology): “Our GitHub repository 1 open-sourced Personalized and Non-personalized DNSMOS P.835 [8] and word accuracy (WAcc) APIs to empower iterative model improvements for teams articipat-ing in the challenge.”
Torchaudio-Squim: Reference-Less Speech Quality and Intelligibility Measures in Torchaudio (2023, background): “Some such as DNSMOS [12] are trained on large-scale crowdsourced data of MOS ratings for speech, while others leverage pre-trained self-supervised models for improved estimation [7].”
Speech Enhancement and Dereverberation With Diffusion-Based Generative Models (2022, methodology): “835 [63] is based on a listening experiment according to ITU-T P.”
CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models (2024, neutral): “NMOS score [53] is used to evaluate the objective quality.”