Input-agreement: a new mechanism for collecting data using human computation games

By Edith Law, Luis von Ahn. 2009

311 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Science Applications, Transportation, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Science Applications

In Transportation

In Evaluation & Benchmarks

Citations in the corpus

WaveNet: A Generative Model for Raw Audio (2016, neutral): “• the MagnaTagATune dataset (Law & Von Ahn, 2009), which consists of about 200 hours of music audio.”
Flock: Hybrid Crowd-Machine Learning Classifiers (2015, background): “Different interfaces may also lead to beneficial outcomes: an input-agreement game, where users are incentivized to describe examples using keywords to each other to see if they were presented the same example, may uncover other features [27].”
Temporal Pooling and Multiscale Learning for Automatic Annotation and Ranking of Music Audio (2011, methodology): “We describe several previous approaches that follow these steps and have been applied to the Magnatune dataset [13] in Section 3.”
Exploring automatic music annotation with "acoustically-objective" tags (2010, unknown): “introduced the MagnaTagatune data set that consists of 6,362 copyright-cleared songs by 269 artists [9].”
The Million Song Dataset (2011, methodology): “This makes it the largest available dataset for tagging evaluation, as compared to Mag-natagatune [9], Swat10K [10] and the Last.”
Social Cues, Social Biases: Stereotypes in Annotations on People Images (2018, background): “Human computation researchers aim for correctness (Law and von Ahn 2011), which is typically evaluated by between-user agreement (Law and von Ahn 2009).”
End-to-end learning for music audio (2014, methodology): “To compare end-to-end learning with the traditional MIR approach of using a mid-level representation of the audio signals, we trained deep CNNs to perform automatic tagging on the Magnatagatune dataset [22].”
Freesound Datasets: A Platform for the Creation of Open Audio Datasets (2017, unknown): “Since gamiﬁcation was a research goal in itself, the annotation tool is well documented [16].”