DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applications

By Wei He, Kai Liu, Jing Liu, Yajuan Lyu, Shiqi Zhao, Xinyan Xiao, Yuan Liu, Yizhong Wang, Hua Wu, Qiaoqiao She and 3 more. 2017

309 citations recorded by the source corpus.

This paper introduces DuReader, a new large-scale, open-domain Chinese machine reading comprehension (MRC) dataset, designed to address real-world MRC. DuReader has three advantages over previous MRC datasets: (1) data sources: questions and documents are based on Baidu Search and Baidu Zhidao; answers are manually generated. (2) question types: it provides rich annotations for more question types, especially yes-no and opinion questions, that leaves more opportunity for the research community. (3) scale: it contains 200K questions, 420K answers and 1M documents; it is the largest Chinese MRC dataset so far. Experiments show that human performance is well above current state-of-the-art baseline systems, leaving plenty of room for the community to make improvements. To help the community make these improvements, both DuReader and baseline systems have been posted online. We also organize a shared competition to encourage the exploration of more models. Since the release of the task, there are significant improvements over the baselines.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

PanGu-α: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation (2021, methodology): “Note that for the DuReader task, we select the Zhidao subset for evaluation in our experiment.”
TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages (2020, background): “With the exception of DuReader, these sets also come with the same lexical overlap caveats as SQuAD.”
The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset (2023, methodology): “DuReader zhs He et al. (2018) Enriched CONLLU Ancora for ML training ca Rodriguez-Penagos and Armentano-Oller (2021a) ESTER fr Galliano et al. (2006) Github code Github on BigQuery Habibi ar El-Haj (2020) HAL fr hal.”
ERNIE 2.0: A Continual Pre-training Framework for Language Understanding (2019, methodology): “Specifically, the following Chinese datasets are chosen to evaluate the performance of ERNIE 2.0 on Chinese tasks: • Machine Reading Comprehension (MRC): CMRC 2018 (Cui et al. 2018), DRCD (Shao et al. 2018), and DuReader (He et al. 2017).”
DRCD: a Chinese Machine Reading Comprehension Dataset (2018, methodology): “We use BiDAF which is implemented by He et al. (2017). We use pre-trained word embedding instead of randomly initialize. The other hyper-parameters remain the same as He et al. (2017). The result of BiDAF model is F1 score 51.”
A Span-Extraction Dataset for Chinese Machine Reading Comprehension (2019, background): “In open-domain reading comprehension, He et al. (2017) propose a large-scale open-domain Chinese machine reading comprehension dataset (DuReader), which contains 200k queries annotated from the user query logs on the search engine.”
Multi-Passage Machine Reading Comprehension with Cross-Passage Answer Verification (2018, methodology): “For DuReader, we follow the preprocessing described in He et al. (2017).”
ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation (2021, positive): “We comprehensively evaluate the ability of ERNIE 3.0 on machine reading comprehension in different aspects, including span-predict reading comprehension (CMRC2018, DuReader, DRCD, DuReaderchecklist), multiple-choice reading comprehension (C3, DuReaderyesno), cloze and completion (CHID, CMRC2019), an”
Multi-style Generative Reading Comprehension (2019, unknown): “DuReader (He et al., 2018), a Chinese multi-document RC dataset, provides the top-10 ranked entire documents from Baidu Search and Zhidao.”
Multilingual Extractive Reading Comprehension by Runtime Machine Translation (2018, methodology): “To alleviate this problem, previous work has aimed at training an RC model for a language of interest, by creating a new largescale dataset for a language of interest (He et al. 2017), or combining a medium-scale dataset with an existing dataset translated from English to the target language (Lee et”
A Comprehensive Overview of Large Language Models (2023, unknown): “0 controllable dataset NLU: NLPCC2014-SC, SE-ABSA16_PHNS, SE-ABSA16_CAME, BDCI2019, EPRSTMT [293], COTE-BD [268], COTE-MFW [268], OCNLI [269], CMNLI [269], OCNLI-FC [293], CLUEWSC [269] CLUEWSC-FC [293], FinRE [270], SanWen [271], AFQMC [269], LCQMC [242], PAWS-X [231], BQ Corpus [272], CSL [269] CS”
C-Pack: Packaged Resources To Advance General Chinese Embedding (2023, background): “In the past few years, the community has put forward essential datasets to study Chinese text embeddings, such as CMNLI (Xu et al., 2020a), DuReader (He et al., 2017), T 2 Ranking (Xie et al., 2023).”
WuDaoCorpora: A super large-scale Chinese corpora for pre-training language models (2021, background): “The quality of NLP models strongly relies on the scale of corpus used for model training (He et al., 2018; Duan et al., 1912; Cui et al., 2019; Qi et al., 2001).”
Probing Prior Knowledge Needed in Challenging Chinese Machine Reading Comprehension (2019, methodology): “Another kind of non-extractive question answering tasks (He et al., 2017) is based on search engines (similar to English MS MARCO (Nguyen et al.”
ERNIE-Gram: Pre-Training with Explicitly N-Gram Masked Language Modeling for Natural Language Understanding (2020, background): “machine reading comprehension (CMRC2018; Cui et al., 2018, DRCD; Shao et al., 2018 and DuReader; He et al., 2018), named entity recognition (MSRA-NER; Gao et al.”
C-Pack: Packed Resources For General Chinese Embeddings (2023, background): “Particularly, the following labeled datasets are included, T 2 - Ranking [60], DuReader [20, 42], mMARCO [8], CMedQA-v2[65], multi-cpr[31], NLI-Zh 9 , cmnli[62] and ocnli[62].”