webMUSHRA — A Comprehensive Framework for Web-based Listening Tests

By Michael Schoeffler, Sarah Bartoschek, Fabian-Robert Stöter, M. Roess, S. Westphal, B. Edler, J. Herre. 2018

262 citations recorded by the source corpus.

For a long time, many popular listening test methods, such as ITU-R BS.1534 (MUSHRA), could not be carried out as web-based listening tests, since established web standards did not support all required audio processing features. With the standardization of the Web Audio API, the required features became available and, therefore, also the possibility to implement a wide range of established methods as web-based listening tests. In order to simplify the implementation of MUSHRA listening tests, the development of webMUSHRA was started. By utilizing webMUSHRA, experimenters can configure web-based MUSHRA listening tests without the need of web programming expertise. Today, webMUSHRA supports many more listening test methods, such as ITU-R BS.1116 and forced-choice procedures. Moreover, webMUSHRA is highly customizable and has been used in many auditory studies for different purposes.

Subfields: Cognitive Neuroscience, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Signal Processing

Citations in the corpus

An Open source Implementation of ITU-T Recommendation P.808 with Validation (2020, background): “CrowdMOS [6] ACR, MUSHRA Y N N Y Y (N=2) QualityCrowd2 [7] ACR Y N N N N BeaqleJS [8] ABX, MUSHRA Y N N N N WESP+ [9] ACR, DCR, PC, Y N N N N DSIS, DSCQS webMUSHRA [10] AB, ABX, Y N N N N MUSHRA, ACR CAQE [11] MUSHRA, PC N Y N Y N P.”
Speech Enhancement and Dereverberation With Diffusion-Based Generative Models (2022, unknown): “Therefore, we conduct a MUSHRA listening experiment [67] with ten participants using the webMUSHRA framework [68].”
Adversarial Attacks Against Automatic Speech Recognition Systems via Psychoacoustic Hiding (2018, methodology): “We used the webMUSHRA implementation, which is available online and was developed by AudioLabs [47].”
Improving Prosody Modelling with Cross-Utterance Bert Embeddings for End-to-End Speech Synthesis (2020, methodology): “Two evaluation metrics were used: the MUSHRA mean opinion score and the ABN choice preference rate [39], For MUSHRA mean opinion test, only single sentence front the testing subset is used in synthesizing test audio.”
SpeedySpeech: Efficient Neural Speech Synthesis (2020, unknown): “We used a setting based on MUSHRA [19, 20]: the participants were shown anonymized outputs of all models and the reference for a given sentence, and they rated them on a ﬁne-grained 100-point scale, visually divided into 5 categories: “Ex-cellent”, “Fair”, “Good”, “Poor” and “Bad”.”