Method for the subjective assessment of intermediate quality level of

2014

Subfields: Evaluation & Benchmarks

Citations in the corpus

• High Fidelity Neural Audio Compression (2022, methodology): “For the subjective tests we follow the MUSHRA protocol (Series, 2014), using both a hidden reference and a low anchor.”
• Speech Resynthesis from Discrete Disentangled Self-Supervised Representations (2021, methodology): “We use a subjective MUSHRA-type listening test [51] to measure the perceived quality of the proposed speech codec with regard to its bitrate constraints.”
• AudioGen: Textually Guided Audio Generation (2022, methodology): “We follow the MUSHRA protocol (Series, 2014), using both a hidden reference and a low anchor.”
• End-to-End Neural Audio Coding in the MDCT Domain (2023, background): “MUSHRA [27] listening tests were also conducted to evaluate the subjective quality of decoded signals.”
• SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models (2023, methodology): “We adopt a crowd-sourced methodology inpspired by MUSHRA protocol (Series, 2014), with a hidden reference but no lowerpass-filtered anchor, for subjective evaluation.”
• TokenSplit: Using Discrete Speech Representations for Direct, Refined, and Transcript-Conditioned Speech Separation and Recognition (2023, methodology): “2), we performed MUSHRA listening tests [33] to measure the subjective quality of our models to provide a more reliable measurement of relative quality.”
• Speech Enhancement for a Noise-Robust Text-to-Speech Synthesis System Using Deep Recurrent Neural Networks (2016, methodology): “We recruited 30 native English speakers to participate on a MUSHRA-style [34] listening test.”
• Temporal Noise Shaping with Companding (2018, methodology): “In the following, we evaluate the impact of companding on perceptual quality with MUSHRA [20] listening tests.”
• Incremental Text to Speech for Neural Sequence-to-Sequence Models using Reinforcement Learning (2020, unknown): “We considered using a MUSHRA test [38].”
• Crowdsourced Pairwise-Comparison for Source Separation Evaluation (2018, background): “However, while crowdsourced MUSHRA-like tests can be a proxy for lab-based MUSHRA tests in some situations, these tests recommend to limit the number of stimuli to only 12 audio signals [7].”
• SpeedySpeech: Efficient Neural Speech Synthesis (2020, unknown): “We used a setting based on MUSHRA [19, 20]: the participants were shown anonymized outputs of all models and the reference for a given sentence, and they rated them on a fine-grained 100-point scale, visually divided into 5 categories: “Ex-cellent”, “Fair”, “Good”, “Poor” and “Bad”.”
• Controllable Emphasis with zero data for text-to-speech (2023, methodology): “We leveraged MUSHRA [31] whenever recordings are available and preference tests otherwise.”
• Speech Enhancement and Dereverberation With Diffusion-Based Generative Models (2022, methodology): “Therefore, we conduct a MUSHRA listening experiment [58] with ten participants using the webMUSHRA”