Effect of trapping questions on the reliability of speech quality judgments in a crowdsourcing paradigm

40 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

An Open source Implementation of ITU-T Recommendation P.808 with Validation (2020, methodology): “It cannot be concluded that the trapping question component is not beneficial as previous studies showed that even its presence has a significant impact on participants’ behavior and encouraging them to provide higher quality responses [14, 20].”
Impact of the Number of Votes on the Reliability and Validity of Subjective Speech Quality Assessment in the Crowdsourcing Approach (2020, background): “.11300v1 [cs.MM] 25 Mar 2020 With the rise of crowdsourcing (CS) micro-task platforms, proposals have been made to transfer listening quality assessments and other assessments of quality to the crowd [4]–[7]. The use of online portals might attract a larger and more diverse group of listeners, thus ”
A Crowdsourcing Extension of the ITU-T Recommendation P.835 with Validation (2020, methodology): “In every crowdsourcing task, it is recommended to include trapping and gold questions [6, 7].”
Subjective Evaluation of Noise Suppression Algorithms in Crowdsourcing (2021, methodology): “The trapping stimuli is an obvious quality control mechanism [13] which asks the participant to select a speciﬁc response to show their attention.”