ViSQOL v3: An Open Source Production Ready Objective Speech and Audio Metric

By Michael Chinen, Felicia S. C. Lim, Jan Skoglund, Nikita Gureev, Feargus O'Gorman, Andrew Hines. 2020

229 citations recorded by the source corpus.

Estimation of perceptual quality in audio and speech is possible using a variety of methods. The combined v3 release of ViSQOL and ViSQOLAudio (for speech and audio, respectively,) provides improvements upon previous versions, in terms of both design and usage. As an open source C++ library or binary with permissive licensing, ViSQOL can now be deployed beyond the research context into production usage. The feedback from internal production teams at Google has helped to improve this new release, and serves to show cases where it is most applicable, as well as to highlight limitations. The new model is benchmarked against real-world data for evaluation purposes. The trends and direction of future work is discussed.

Subfields: Artificial Intelligence, Evaluation & Benchmarks, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Evaluation & Benchmarks

In Speech & Audio

In Signal Processing

Citations in the corpus

SoundStream: An End-to-End Neural Audio Codec (2021, positive): “We choose the freely available and recently open-sourced ViSQOL [56], [57] metric, which has previously shown comparable performance to POLQA.”
High Fidelity Neural Audio Compression (2022, methodology): “Neural based audio codecs have been recently proposed and demonstrated promising results (Kleijn et al., 2018; Valin & Skoglund, 2019b; Lim et al., 2020; Kleijn et al., 2021; Zeghidour et al., 2021; Omran et al., 2022; Lin et al., 2022; Jayashankar et al., 2022; Li et al.; Jiang et al., 2022),…”
High-Fidelity Audio Compression with Improved RVQGAN (2023, methodology): “For the first, we use periods of [2, 3, 5, 7, 11], and for the second, we use window lengths [2048, 1024, 512], with a hop-length that is 1/4 the window length.”
Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis (2023, methodology): “To assess perceptual quality, we also utilize ViSQOL (Chinen et al., 2020) in audio-mode, which operates in the full band.”
AudioLM: A Language Modeling Approach to Audio Generation (2022, background): “score [49], [50], a computational proxy for perceived similarity between a reference audio and its reconstruction.”
AudioGen: Textually Guided Audio Generation (2022, methodology): “To assess the quality of the audio representation we report three encoding metrics namely Scale-Invariant Signal-to-Noise Ration (SI-SNR), ViSQOL (Chinen et al., 2020), and bitrate.”
EasyCom: An Augmented Reality Dataset to Support Algorithms for Easy Communication in Noisy Environments (2021, methodology): “Finally, the last metric used in the evaluation is the Virtual Speech Quality Objective Listener (ViSQOL) version 3 metric [29], which has been shown to have higher correlation to speech quality than PESQ on several datasets [32].”
TokenSplit: Using Discrete Speech Representations for Direct, Refined, and Transcript-Conditioned Speech Separation and Recognition (2023, methodology): “To measure signal quality, we use scale-invariant signal-to-noise ratio improvement (SI-SNRi) [30], Virtual Speech Quality Objective Listener (ViSQOL) [31], and the non-intrusive Deep Noise Suppression Mean Opinion Score (DNSMOS) [32].”
Disentangling speech from surroundings in a neural audio codec (2022, methodology): “[29] A. Hines, J. Skoglund, A. C. Kokaram, and N. Harte, “ViSQOL: An objective speech quality model,” EURASIP Journal on Audio, Speech, and Music Processing, p. 13, 2015.”
LMCodec: A Low Bitrate Speech Codec with Causal Transformer Models (2023, methodology): “Despite neural speech codecs metrics ViSQOL and WARP-Q indicating worse performance at about 4-6 quantizers, our listening test shows very high quality audio results with small number of quantizers.”