Cross-Lingual Machine Reading Comprehension

By Yiming Cui, Wanxiang Che, Ting-Wei Liu, Bing Qin, Shijin Wang, Guoping Hu. 2019

48 citations recorded by the source corpus.

Though the community has made great progress on Machine Reading Comprehension (MRC) task, most of the previous works are solving English-based MRC problems, and there are few efforts on other languages mainly due to the lack of large-scale training data.In this paper, we propose Cross-Lingual Machine Reading Comprehension (CLMRC) task for the languages other than English. Firstly, we present several back-translation approaches for CLMRC task which is straightforward to adopt. However, to exactly align the answer into source language is difficult and could introduce additional noise. In this context, we propose a novel model called Dual BERT, which takes advantage of the large-scale training data provided by rich-resource language (such as English) and learn the semantic relations between the passage and question in bilingual context, and then utilize the learned knowledge to improve reading comprehension performance of low-resource language. We conduct experiments on two Chinese machine reading comprehension datasets CMRC 2018 and DRCD. The results show consistent and significant improvements over various state-of-the-art systems by a large margin, which demonstrate the potentials in CLMRC task. Resources available: https://github.com/ymcui/Cross-Lingual-MRC

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jimmy J. Lin
2. Zhe Gan
3. Rodrigo Nogueira
4. Andrew Yates
5. Jingjing Liu
6. Shuohang Wang
7. Jianfeng Gao
8. Furu Wei

In Computer Vision and Pattern Recognition

1. Jimmy J. Lin
2. Zhe Gan
3. Jingjing Liu
4. Shuohang Wang
5. Andrew Yates
6. Rodrigo Nogueira
7. Siqi Sun
8. Yuwei Fang

In Natural Language Processing

1. Jimmy J. Lin
2. Rodrigo Nogueira
3. Zhe Gan
4. Andrew Yates
5. Shuohang Wang
6. Jingjing Liu
7. Joachim Daiber
8. Niklas Muennighoff

Citations in the corpus

• Pretrained Transformers for Text Ranking: BERT and Beyond (2020, background): “were confirmed by MacAvaney et al. [2020e]. The second point in Google’s blog post likely refers to multi-lingual question answering, which recently has benefited from the introduction of new datasets [Cui et al., 2019, Liu et al., 2019a]. Although some of the early neural ranking approaches did explo”
• MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering (2020, background): “…transfer learning and zero/few-shot methods narrow the multilingual performance gap with English (Conneau et al., 2020; Lee and Lee, 2019; Cui et al., 2019a; Lewis et al., 2020), their real progress is hard to measure without challenging, realistic, and linguistically diverse evaluation…”
• Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model (2024, methodology): “Translate-train, translate-test (Asai et al., 2018; Cui et al., 2019; Jundi and Lapesa, 2022), or language pivots (Montero et al., 2022) are common techniques employing translation models to bridge language gaps between the model and its target language.”
• FILTER: An Enhanced Fusion Method for Cross-lingual Language Understanding (2020, methodology): “Results of mBERT, MMTE, XLM and XLM-R are from XTREME (Hu et al. 2020). mtl denotes translate-train in multi-task version.”