ViSQOL: an objective speech quality model

By Andrew Hines, Jan Skoglund, Anil C. Kokaram, Naomi Harte. 2015

202 citations recorded by the source corpus.

This paper presents an objective speech quality model, ViSQOL, the Virtual Speech Quality Objective Listener. It is a signal-based, full-reference, intrusive metric that models human speech quality perception using a spectro-temporal measure of similarity between a reference and a test speech signal. The metric has been particularly designed to be robust for quality issues associated with Voice over IP (VoIP) transmission. This paper describes the algorithm and compares the quality predictions with the ITU-T standard metrics PESQ and POLQA for common problems in VoIP: clock drift, associated time warping, and playout delays. The results indicate that ViSQOL and POLQA significantly outperform PESQ, with ViSQOL competing well with POLQA. An extensive benchmarking against PESQ, POLQA, and simpler distance metrics using three speech corpora (NOIZEUS and E4 and the ITU-T P.Sup. 23 database) is also presented. These experiments benchmark the performance for a wide range of quality impairments, including VoIP degradations, a variety of background noise types, speech enhancement methods, and SNR levels. The results and subsequent analysis show that both ViSQOL and POLQA have some performance weaknesses and under-predict perceived quality in certain VoIP conditions. Both have a wider application and robustness to conditions than PESQ or more trivial distance metrics. ViSQOL is shown to offer a useful alternative to POLQA in predicting speech quality in VoIP scenarios.

Subfields: Computational Mechanics, Evaluation & Benchmarks, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Evaluation & Benchmarks

In Speech & Audio

In Signal Processing

Citations in the corpus

FunCodec: A Fundamental, Reproducible and Integrable Open-Source Toolkit for Neural Speech Codec (2023, methodology): “It is worth noting We adopt the Virtual Speech Quality Objective Listener score (ViSQOL) [33] as the primary evaluation metric, which ranges from 1 to 5, with higher scores indicating better quality.”
DelightfulTTS 2: End-to-End Speech Synthesis with Adversarial Vector-Quantized Auto-Encoders (2022, methodology): “For objective metrics, we report results measured by means of ViSQOL [25] in ablation studies.”
Jitter Buffer Compensation in Voice over IP Quality Estimation (2019, methodology): “The POLQA metric results vary heavily with certain delay lengths, which perform well when conditions fit this model, while giving continuously errors with unsuitable circumstances [6].”
Disentangling speech from surroundings in a neural audio codec (2022, methodology): “[29] A. Hines, J. Skoglund, A. C. Kokaram, and N. Harte, “ViSQOL: An objective speech quality model,” EURASIP Journal on Audio, Speech, and Music Processing, p. 13, 2015.”
Measuring and monitoring speech quality for voice over IP with POLQA, viSQOL and p.563 (2015, methodology): “ViSQOL [18] evolved from NSIM (Neurogram Similarity Index Measure), a tool developed by Hines and Harte [19] to predict speech intelligibility for hearing-impaired listeners.”
An Overview of Deep-Learning-Based Audio-Visual Speech Enhancement and Separation (2020, background): “ViSQOL [93], [94] 2012 Specifically designed for voice over [55], [183] IP transmission”
AMBIQUAL - a full reference objective quality metric for ambisonic spatial audio (2018, background): “C. ViSQOL and NSIM ViSQOL Speech [17] and ViSQOLAudio [18] are full reference objective metrics for measuring speech quality and audio quality respectively.”
Dnsmos P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressors (2021, background): “Conventional objective speech quality metrics such as Perceptual Evaluation of Speech Quality (PESQ) [4], Perceptual Objective Listening Quality Analysis (POLQA) [5], VisQOL [6] and Signal to Distortion Ratio (SDR) are widely used to evaluate Speech Enhancement (SE) algorithms optimized for human pe”
A scalable noisy speech dataset and online subjective test framework (2019, methodology): “Most of the published SE methods are evaluated using objective measures such as Perceptual Evaluation of Speech Quality (PESQ) [7], Perceptual Objective Listening Quality Analysis (POLQA) [8] and Virtual Speech Quality Objective Listener (ViSQOL) [9].”
The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Speech Quality and Testing Framework (2020, background): “…literature report experimental results based on objective speech quality metrics such as Perceptual Evaluation of Speech Quality (PESQ) 1 , Perceptual Objective Listening Quality Analysis (POLQA) [11], Virtual Speech Quality Objective Listener (ViSQOL) [12], Speech to Distortion Ratio (SDR).”
FoundationTTS: Text-to-Speech for ASR Customization with Generative Language Model (2023, methodology): “We can observe that ViSQOL score of fine-grained codec is +0.7 over coarse-grained codec, but coarse-grained codec using only 7.9% bitrates compared with fine-grained codec, higher bitrates of fine-grained codec lead to better audio reconstruct quality and lower bitrates of coarse-grained codec resu”
EasyCom: An Augmented Reality Dataset to Support Algorithms for Easy Communication in Noisy Environments (2021, methodology): “Finally, the last metric used in the evaluation is the Virtual Speech Quality Objective Listener (ViSQOL) version 3 metric [29], which has been shown to have higher correlation to speech quality than PESQ on several datasets [32].”
Performance Study of a Convolutional Time-Domain Audio Separation Network for Real-Time Speech Denoising (2020, result): “Nevertheless, with this in mind, and taking the estimated scores with a grain of salt, we also compare the denoising methods using ViSQOL [24].”
Objective Assessment of Perceptual Audio Quality Using ViSQOLAudio (2017, background): “ViSQOL [7] is a speech quality model that was later adapted to function as a perceptual audio quality model, yielding a prototype that delivered promising results [4].”
The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Testing Framework, and Challenge Results (2020, methodology): “com/microsoft/MS-SNSD Evaluation of Speech Quality (PESQ) [11], Perceptual Objective Listening Quality Analysis (POLQA) [12], Virtual Speech Quality Objective Listener (ViSQOL) [13], Speech to Distortion Ratio (SDR).”