BiPaR: A Bilingual Parallel Dataset for Multilingual and Cross-lingual Reading Comprehension on Novels
By Yimin Jing, Deyi Xiong, Yan Zhen. 2019
34 citations recorded by the source corpus.
This paper presents BiPaR, a bilingual parallel novel-style machine reading comprehension (MRC) dataset, developed to support multilingual and cross-lingual reading comprehension. The biggest difference between BiPaR and existing reading comprehension datasets is that each triple (Passage, Question, Answer) in BiPaR is written parallelly in two languages. We collect 3,667 bilingual parallel paragraphs from Chinese and English novels, from which we construct 14,668 parallel question-answer pairs via crowdsourced workers following a strict quality control procedure. We analyze BiPaR in depth and find that BiPaR offers good diversification in prefixes of questions, answer types and relationships between questions and passages. We also observe that answering questions of novels requires reading comprehension skills of coreference resolution, multi-sentence reasoning, and understanding of implicit causality, etc. With BiPaR, we build monolingual, multilingual, and cross-lingual MRC baseline models. Even for the relatively simple monolingual MRC on this dataset, experiments show that a strong BERT baseline is over 30 points behind human in terms of both EM and F1 score, indicating that BiPaR provides a challenging testbed for monolingual, multilingual and cross-lingual MRC on novels. The dataset is available at https://multinlp.github.io/BiPaR/.
Subfields: Artificial Intelligence, General Social Sciences, Evaluation & Benchmarks, Machine Translation, Question Answering & Reading Comprehension
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In General Social Sciences
In Evaluation & Benchmarks
Citations in the corpus
- MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering (2020, background): “, 2019b) and BiPar (Jing et al., 2019) present parallel, cross-lingual QA dataset between English and Chinese.”
- Beyond Leaderboards: A survey of methods for revealing weaknesses in Natural Language Inference data and models (2020, background): “…et al., 2018), ProPara (Dalvi et al., 2018), MedHop (Welbl et al., 2018), OBQA (Mihaylov et al., 2018), BioASQ (Kamath et al., 2018) 2019 BiPaR (Jing et al., 2019), NaturalQ (Kwiatkowski et al., 2019), ROPES (Lin et al., 2019), SherLIiC (Schmitt and Schütze, 2019), CLUTRR (Sinha et al.,…”