DRCD: a Chinese Machine Reading Comprehension Dataset

By Chih-Chieh Shao, Trois Liu, Yuting Lai, Yiying Tseng, Sam S. Tsai. 2018

142 citations recorded by the source corpus.

In this paper, we introduce DRCD (Delta Reading Comprehension Dataset), an open domain traditional Chinese machine reading comprehension (MRC) dataset. This dataset aimed to be a standard Chinese machine reading comprehension dataset, which can be a source dataset in transfer learning. The dataset contains 10,014 paragraphs from 2,108 Wikipedia articles and 30,000+ questions generated by annotators. We build a baseline model that achieves an F1 score of 89.59%. F1 score of Human performance is 93.30%.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

PanGu-α: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation (2021, methodology): “Regarding CMRC2018, DRCD and WebQA tasks of PanGu-α 13B, the few-shot performance surpasses zero-shot by more than 10 points, demonstrating that PanGu-α 13B has superior in-context learning ability.”
TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages (2020, methodology): “, 2017) and DRCD (Shao et al., 2018) in Chinese, French/Japanese evaluation sets for SQuAD created via translation (Asai et al.”
Data Augmentation for BERT Fine-Tuning in Open-Domain Question Answering (2019, methodology): “There are two additional characteristics to note about our data augmentation techniques: The most salient characteristic is that SQuAD, CMRC, and DRCD all have source answers drawn from Wikipedia (English or Chinese), while TriviaQA includes web pages as well as Wikipedia.”
MLQA: Evaluating Cross-lingual Extractive Question Answering (2019, neutral): “There are several Chinese datasets, such as DUReader (He et al., 2018), CMRC (Cui et al., 2019b) and DRCD (Shao et al., 2018).”
Neural Approaches to Conversational AI (2018, neutral): “, 2017), DRCD (Shao et al., 2018)), news and other articles (CNN/Daily Mail (Hermann et al.”
ERNIE 2.0: A Continual Pre-training Framework for Language Understanding (2019, methodology): “For MRC task, CMRC 2018 [27], DRCD [28], DuReader [29] are used as test datasets.”
BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis (2019, background): “…and objective texts, e.g., Wikipedia (WikiReading (Hewlett et al., 2016), SQuAD (Rajpurkar et al., 2016, 2018), Wiki-Hop (Welbl et al., 2018), DRCD (Shao et al., 2018), QuAC (Choi et al., 2018), HotpotQA (Yang et al., 2018)) news and other articles (CNN/Daily Mail (Hermann et al., 2015), NewsQA…”
A Span-Extraction Dataset for Chinese Machine Reading Comprehension (2019, background): “Shao et al. (2018) proposed a reading comprehension dataset in Traditional Chinese.”
ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation (2021, positive): “We comprehensively evaluate the ability of ERNIE 3.0 on machine reading comprehension in different aspects, including span-predict reading comprehension (CMRC2018, DuReader, DRCD, DuReaderchecklist), multiple-choice reading comprehension (C3, DuReaderyesno), cloze and completion (CHID, CMRC2019), an”
ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation (2021, background): “• Machine Reading Comprehension (MRC) : DRCD [86], DuReader [87], Dureader robust [88], Dureader checklist , Dureader yesno 18 , C 3 [89], CMRC 2018 [90].”
How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models (2020, methodology): “For ZH, we use the Delta Reading Comprehension Dataset (DRCD; Shao et al., 2019), which was crowdsourced from Chinese Wikipedia articles.”
Zero-shot Reading Comprehension by Cross-lingual Transfer Learning with Multi-lingual Language Representation Model (2019, methodology): “Although some efforts have been made to create RC datasets for Chinese (He et al., 2018; Shao et al., 2018) and Korean (Seungyoung Lim, 2018), it is not feasible to collect RC datasets for every language since annotation efforts to collect a new RC dataset are often far from trivial.”
Probing Prior Knowledge Needed in Challenging Chinese Machine Reading Comprehension (2019, methodology): “We have seen the construction of span-style Chinese machine reading comprehension datasets (Cui et al., 2016, 2018b,a; Shao et al., 2018), using Chinese news reports, books, and Wikipedia articles as source documents, similar to their English counterparts CNN/Daily Mail (Hermann et al., 2015),…”
TextBrewer: An Open-Source Knowledge Distillation Toolkit for Natural Language Processing (2020, methodology): “For Chinese datasets, we use XNLI (Conneau et al., 2018), LCQMC (Liu et al., 2018), CMRC 2018 (Cui et al., 2019b) and DRCD (Shao et al., 2018).”
SPARTA: Efficient Open-Domain Question Answering via Sparse Transformer Matching Retrieval (2020, methodology): “OpenSQuAD Model F1 EM Dr.QA(Chen et al., 2017) - 29.8 R 3 (Wang et al., 2018) 37.5 29.1 Par. ranker (Lee et al., 2018) - 30.2 MINIMAL (Min et al., 2018) 42.5 32.7 DenSPI-hybrid (Seo et al., 2019) 44.4 36.2 BERTserini (Yang et al., 2019a) 46.1 38.6 RE 3 (Hu et al., 2019) 50.2 41.9 Multi-passage (Wang”
A Sentence Cloze Dataset for Chinese Machine Reading Comprehension (2020, methodology): “In span-extraction MRC, Cui et al. (2019b) proposed CMRC 2018 dataset for Simpliﬁed Chinese, and Shao et al. (2018) proposed DRCD dataset for Traditional Chinese, similar to the popular dataset SQuAD (Rajpurkar et al., 2016).”
Pre-Training with Whole Word Masking for Chinese BERT (2019, methodology): “We mainly test on three datasets: CMRC 2018 (Cui et al., 2018), DRCD (Shao et al., 2018), CJRC. • CMRC 2018 : A span-extraction machine reading comprehension dataset, which is similar to SQuAD (Rajpurkar et al., 2016) that extract a passage span for the given question.”