Evaluation of Algorithms Using Games: The Case of Music Tagging

By Edith Law, Kris West, Michael I. Mandel, Mert Bay, J. Stephen Downie. 2009

323 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Yoshua Bengio
2. Yu Zhang
3. Jiahui Yu
4. Bo Li
5. Quoc V. Le
6. William Chan
7. Wei Han
8. Xu Tan

Citations in the corpus

• SoundStream: An End-to-End Neural Audio Codec (2021, methodology): “For music, we use the MagnaTagATune dataset [52].”
• Qwen3-Omni Technical Report (2025, unknown): “On GTZAN, MTG-Jamendo, and MagnaTagATune, the scores of Qwen3-Omni-Instruct also significantly surpass other audio language models, including Gemini-2.”
• MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training (2023, methodology): “For music tagging, we utilise the MagnaTagATune (MTT) (Law et al., 2009) and MTGJamendo (Bogdanov et al.”
• Automatic Tagging Using Deep Convolutional Neural Networks (2016, methodology): “In crowd-sourced music tag datasets [2,13], most of the tags are false (0) for most of the clips, which makes accuracy or mean square error inappropriate as a measure.”
• Evaluation of CNN-based Automatic Music Tagging Models (2020, methodology): “MagnaTagATune (MTAT) [14] is one of the most commonly used datasets for benchmarking automatic music tagging systems.”
• Noise2Music: Text-conditioned Music Generation with Diffusion Models (2023, methodology): “MTAT contains 25,863 music clips, 21,638 of which are associated with multiple tags from a vocabulary of 188 music tags.”
• Wavelet Networks: Scale Equivariant Learning From Raw Waveforms (2020, methodology): “The MagnaTagATune (MTAT) dataset [32] consists of 25879 audio clips with a total of 170 hours of audio, along with several per-song tags.”
• Sample-level Deep Convolutional Neural Networks for Music Auto-tagging Using Raw Waveforms (2017, methodology): “We evaluate the proposed model on two datasets, Mag-natagatune dataset (MTT) [16] and Million song dataset (MSD) annotated with the Last.”
• Freesound Datasets: A Platform for the Creation of Open Audio Datasets (2017, unknown): “• The MagnaTagATune [17] dataset includes music data released under Creative Commons (CC) licenses, which simplifies data sharing, and annotations (tags and similarity) were made by engaging users in playing a game.”
• The MTG-Jamendo Dataset for Automatic Music Tagging (2019, methodology): “In the context of auto-tagging, recent studies commonly use three datasets (Law et al., 2009; BertinMahieux et al., 2011; Defferrard et al., 2017) summarized in Table 1 and known for their limitations.”
• FMA: A Dataset for Music Analysis (2016, background): “Looking at Table 1, the well-known MagnaTagATune [20] and the Million Song Dataset (MSD) [3] as well as the newer AudioSet [10] and AcousticBrainz [32] appear as contenders for a large-scale reference dataset.”
• Fréchet Audio Distance: A Metric for Evaluating Music Enhancement Algorithms (2018, methodology): “For our experiments, we use the Magnatagatune dataset [10], which contains 600 hours of music samples at 16 kHz.”
• Human computation: a survey and taxonomy of a growing field (2011, background): “[34] Most other papers using the term do not define it explicitly.”
• Temporal Pooling and Multiscale Learning for Automatic Annotation and Ranking of Music Audio (2011, methodology): “This dataset was used in the MIREX 2009 contest on audio tag classification [14].”
• Multi-Level and Multi-Scale Feature Aggregation Using Pretrained Convolutional Neural Networks for Music Auto-Tagging (2017, methodology): “To evaluate the proposed architecture, we primarily used the MagnaTagATune (MTAT) dataset [14] and Million Song Dataset”