MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languages

By Jack G. M. FitzGerald, C. Hench, Charith Peris, S. Mackie, Kay Rottmann, A. Sánchez, Aaron Nash, Liam Urbach, Vishesh Kakarala, Richa Singh and 6 more. 2022

219 citations recorded by the source corpus.

We present the MASSIVE dataset–Multilingual Amazon Slu resource package (SLURP) for Slot-filling, Intent classification, and Virtual assistant Evaluation. MASSIVE contains 1M realistic, parallel, labeled virtual assistant utterances spanning 51 languages, 18 domains, 60 intents, and 55 slots. MASSIVE was created by tasking professional translators to localize the English-only SLURP dataset into 50 typologically diverse languages from 29 genera. We also present modeling results on XLM-R and mT5, including exact match accuracy, intent classification accuracy, and slot-filling F1 score. We have released our dataset, modeling code, and models publicly.

Subfields: Artificial Intelligence, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model (2022, methodology): “Embedding classification performance on MASSIVE (FitzGerald et al., 2022) scored using accuracy”
NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models (2024, unknown): “The classification datasets that we use are as follows: AmazonReviews (McAuley & Leskovec, 2013a), AmazonCounterfactual (O’Neill et al., 2021), Banking77 (Casanueva et al., 2020), Emotion (Saravia et al., 2018), IMDB (Maas et al., 2011), MTOPDomain/MTOPIntent (Li et al., 2021), ToxicConversations (A”
Layer-wise Representation Dynamics: An Empirical Investigation Across Embedders and Base LLMs (2026, unknown): “…Accuracy EmotionClassification [58] Classification Accuracy ImdbClassification [41] Classification Accuracy MassiveIntentClassification [16] Classification Accuracy MassiveScenarioClassification [16] Classification Accuracy MTOPDomainClassification [36] Classification Accuracy…”
Beyond Matryoshka: Revisiting Sparse Coding for Adaptive Representation (2025, unknown): “• MassiveScenario (FitzGerald et al., 2022): A collection of Amazon Alexa virtual assistant utterances annotated with the associated intent.”