Making a MIRACL: Multilingual Information Retrieval Across a Continuum of Languages

By Xinyu Crystina Zhang, Nandan Thakur, Odunayo Ogundepo, Ehsan Kamalloo, David Alfonso-Hermelo, Xiaoguang Li, Qun Liu, Mehdi Rezagholizadeh, Mehdi Rezagholizadeh, Jimmy J. Lin. 2022

71 citations recorded by the source corpus.

MIRACL (Multilingual Information Retrieval Across a Continuum of Languages) is a multilingual dataset we have built for the WSDM 2023 Cup challenge that focuses on ad hoc retrieval across 18 different languages, which collectively encompass over three billion native speakers around the world. These languages have diverse typologies, originate from many different language families, and are associated with varying amounts of available resources -- including what researchers typically characterize as high-resource as well as low-resource languages. Our dataset is designed to support the creation and evaluation of models for monolingual retrieval, where the queries and the corpora are in the same language. In total, we have gathered over 700k high-quality relevance judgments for around 77k queries over Wikipedia in these 18 languages, where all assessments have been performed by native speakers hired by our team. Our goal is to spur research that will improve retrieval across a continuum of languages, thus enhancing information access capabilities for diverse populations around the world, particularly those that have been traditionally underserved. This overview paper describes the dataset and baselines that we share with the community. The MIRACL website is live at http://miracl.ai/.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Information Retrieval & Search, Machine Translation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Niklas Muennighoff
2. Ryohei Sasano
3. Hayato Tsukagoshi
4. Nouamane Tazi
5. Nils Reimers
6. L. Magne
7. Hong-Liu Cao
8. Iryna Gurevych

In Computer Vision and Pattern Recognition

1. Niklas Muennighoff
2. Hayato Tsukagoshi
3. Ryohei Sasano
4. Nils Reimers
5. Iryna Gurevych

In Information Retrieval & Search

1. Niklas Muennighoff
2. Hayato Tsukagoshi
3. Ryohei Sasano
4. Nils Reimers

Citations in the corpus

• MTEB: Massive Text Embedding Benchmark (2022, methodology): “It should be easy to extend MTEB with datasets, such as CodeSearchNet (Husain et al., 2019), TyDI QA (Clark et al., 2020), XOR QA (Asai et al., 2020) or MIRACL (Zhang et al., 2022).”
• Training Sparse Mixture Of Experts Text Embedding Models (2025, unknown): “We train on the training sets of BEIR and MIRACL as well as SQuAD and Stackoverflow.”
• HAKARI-Bench: A Lightweight Benchmark for Comparing Retrieval Architectures and Efficiency Settings under Unified Conditions (2026, unknown): “For multilingual monolingual retrieval, MIRACL (Zhang et al., 2023) is widely used; for short-query passage retrieval, MS MARCO (Bajaj et al., 2016); and for domain specialization, code retrieval CoIR (Li et al., 2024), long-document retrieval LongEmbed (Zhu et al., 2024), instruction-following retr”
• Redundancy, Isotropy, and Intrinsic Dimensionality of Prompt-based Text Embeddings (2025, unknown): “For the evaluation datasets, we use MIRACL (Zhang et al., 2022), Quora, 4 HotpotQA (Yang et al., 2018), DBPedia (Hasibi et al., 2017), Natural Questions (Kwiatkowski et al., 2019), and MS MARCO (Nguyen et al., 2016).”
• Ruri: Japanese General Text Embeddings (2024, methodology): “For reranker training, we utilized existing Japanese retrieval and QA datasets, including JSQuAD (Kurihara et al., 2022), JaQuAD (So et al., 2022), JQaRA (Tateno, 2024b), MKQA (Long-pre et al., 2021), Mr. TyDi (Zhang et al., 2021b), and MIRACL (Zhang et al., 2022).”