XQA: A Cross-lingual Open-domain Question Answering Dataset

By Jiahua Liu, Yankai Lin, Zhiyuan Liu, Maosong Sun. 2019

99 citations recorded by the source corpus.

Open-domain question answering (OpenQA) aims to answer questions through text retrieval and reading comprehension. Recently, lots of neural network-based models have been proposed and achieved promising results in OpenQA. However, the success of these models relies on a massive volume of training data (usually in English), which is not available in many other languages, especially for those low-resource languages. Therefore, it is essential to investigate cross-lingual OpenQA. In this paper, we construct a novel dataset XQA for cross-lingual OpenQA research. It consists of a training set in English as well as development and test sets in eight other languages. Besides, we provide several baseline systems for cross-lingual OpenQA, including two machine translation-based methods and one zero-shot cross-lingual method (multilingual BERT). Experimental results show that the multilingual BERT model achieves the best results in almost all target languages, while the performance of cross-lingual OpenQA is still much lower than that of English. Our analysis indicates that the performance of cross-lingual OpenQA is related to not only how similar the target language and English are, but also how difficult the question set of the target language is. The XQA dataset is publicly available at http://github.com/thunlp/XQA.

Subfields: Artificial Intelligence, Information Systems, Information Retrieval & Search, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Information Retrieval & Search

In Natural Language Processing

Citations in the corpus

TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages (2020, background): “For example, in XQA (Liu et al., 2019a) and XCMRC (Liu et al., 2019b), statements phrased syntactically as questions ( Did you know that is the largest stringray?”
MLQA: Evaluating Cross-lingual Extractive Question Answering (2019, background): “Recognising this need, several cross-lingual datasets have recently been assembled (Asai et al., 2018; Liu et al., 2019a).”
Language Models are Multilingual Chain-of-Thought Reasoners (2022, methodology): “…of existing models, there has been work introducing benchmarks on various multilingual tasks, including cross-lingual question answering (Liu et al., 2019; Clark et al., 2020), natural language inference (Conneau et al., 2018) and bilingual lexicon induction (Lample et al., 2018), as…”
Bridging the domain gap in cross-lingual document classification (2019, methodology): “standing has also been applied to other NLP tasks, with datasets available in dependency parsing (Nivre et al., 2016), natural language inference (XNLI) (Conneau et al., 2018) and question answering (Liu et al. (2019)). Cross-lingual methods gained popularity with the advent of cross-lingual word em”
Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering (2021, background): “7 Free-form A tweet from Twitter XQA [146] Wikipedia 90.”
MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering (2020, background): “Existing training and evaluation sets are oriented to “extrac-tive” QA, providing speciﬁc passages and passage-dependent answer annotations (Clark et al., 2020; Lewis et al., 2020; Artetxe et al., 2020b; Liu et al., 2019a).”