MMCR4NLP: Multilingual Multiway Corpora Repository for Natural Language Processing
By Raj Dabre, Sadao Kurohashi. 2017
14 citations recorded by the source corpus.
Multilinguality is gradually becoming ubiquitous in the sense that more and more researchers have successfully shown that using additional languages help improve the results in many Natural Language Processing tasks. Multilingual Multiway Corpora (MMC) contain the same sentence in multiple languages. Such corpora have been primarily used for Multi-Source and Pivot Language Machine Translation but are also useful for developing multilingual sequence taggers by transfer learning. While these corpora are available, they are not organized for multilingual experiments and researchers need to write boilerplate code every time they want to use said corpora. Moreover, because there is no official MMC collection it becomes difficult to compare against existing approaches. As such we present our work on creating a unified and systematically organized repository of MMC spanning a large number of languages. We also provide training, development and test splits for corpora where official splits are unavailable. We hope that this will help speed up the pace of multilingual NLP research and ensure that NLP researchers obtain results that are more trustable since they can be compared easily. We indicate corpora sources, extraction procedures if any and relevant statistics. We also make our collection public for research purposes.
Subfields: Artificial Intelligence, Natural Language Processing, Evaluation & Benchmarks, Machine Translation
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Juan Pino
- Graham Neubig
- Philipp Koehn
- Changhan Wang
- Jiatao Gu
- Felix Stahlberg
- Yonatan Belinkov
- Chenhui Chu
In Natural Language Processing
- Philipp Koehn
- Graham Neubig
- Yonatan Belinkov
- Nadir Durrani
- Chenhui Chu
- Raj Dabre
- Rico Sennrich
- Jiatao Gu
In Evaluation & Benchmarks
- Marco Turchi
- Matteo Negri
- Ondrej Bojar
- Jan Niehues
- Marcello Federico
- Christian Federmann
- Nadir Durrani
- Roldano Cattoni
In Machine Translation
Citations in the corpus
- A Survey of Multilingual Neural Machine Translation (2019, background): “The MMCR4NLP project (Dabre and Kurohashi, 2017) compiles language family grouped multi-source corpora and provides standard splits.”
- FINDINGS OF THE IWSLT 2020 EVALUATION CAMPAIGN (2020, background): “These include Global Voices, News Commentary, and Ubuntu corpora from OPUS Tiedemann (2012); OpenSubtitles (Lison and Tiedemann, 2016); TED talks (Dabre and Kurohashi, 2017); Wikipedia (Chu et al., 2014, 2015); Wiktionary.org; and WikiMatrix (Schwenk et al., 2019).”