Dataset for the First Evaluation on Chinese Machine Reading Comprehension

By Yiming Cui, Ting-Wei Liu, Z. Chen, Wentao Ma, Shijin Wang, Guoping Hu. 2018

24 citations recorded by the source corpus.

Machine Reading Comprehension (MRC) has become enormously popular recently and has attracted a lot of attention. However, existing reading comprehension datasets are mostly in English. To add diversity in reading comprehension datasets, in this paper we propose a new Chinese reading comprehension dataset for accelerating related research in the community. The proposed dataset contains two different types: cloze-style reading comprehension and user query reading comprehension, associated with large-scale training data as well as human-annotated validation and hidden test set. Along with this dataset, we also hosted the first Evaluation on Chinese Machine Reading Comprehension (CMRC-2017) and successfully attracted tens of participants, which suggest the potential impact of this dataset.

Subfields: Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

1. Zhuosheng Zhang
2. Ting-Wei Liu
3. Zhao Hai
4. Dong Yu
5. Dian Yu
6. Kai Sun
7. Hai Zhao
8. Wanxiang Che

In Evaluation & Benchmarks

1. Dong Yu
2. Qun Liu
3. Claire Cardie
4. Yi Liao
5. Shijin Wang
6. Xin Jiang
7. Wanxiang Che
8. Yiming Cui

Citations in the corpus

• PanGu-α: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation (2021, methodology): “Cloze and completion tasks, including WPLC, CHID [30], PD&CFT [31], CMRC2017 [32], and CMRC2019 [33].”
• A Span-Extraction Dataset for Chinese Machine Reading Comprehension (2019, background): “Later, Cui et al. (2018) propose another dataset, which is gathered from children’s reading material.”
• Effective Subword Segmentation for Text Comprehension (2018, background): “Chinese Machine Reading Comprehension (CMRC-2017) [14], People’s Daily (PD) [11], Children Fairy Tales (CFT) [11] and two English ones, Children’s Book Test (CBT) [20], the Stan-”
• A Sentence Cloze Dataset for Chinese Machine Reading Comprehension (2020, background): “, 2016) Web NQ Entity - 42K CMRC 2017 (Cui et al., 2018) News Cloze&NQ Word - 364K DuReader (He et al.”
• Effective Character-Augmented Word Embedding for Machine Reading Comprehension (2018, methodology): “Since the CFT dataset 2 In the test set of CMRC-2017 and human evaluation test set (Test-human) of CFT, questions are further processed by human and the pattern of them may not be in accordance with the auto-generated questions, so it may be harder for machine to answer.”
• Probing Prior Knowledge Needed in Challenging Chinese Machine Reading Comprehension (2019, methodology): “We have seen the construction of span-style Chinese machine reading comprehension datasets (Cui et al., 2016, 2018b,a; Shao et al., 2018), using Chinese news reports, books, and Wikipedia articles as source documents, similar to their English counterparts CNN/Daily Mail (Hermann et al., 2015),…”
• ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation (2021, unknown): “): PD&CFT [81], CMRC2017 [82], CMRC2019 [83], CHID [84], CHID-FC [68], WPLC [85].”
• A Comprehensive Overview of Large Language Models (2023, unknown): “Commonsense Reasoning WinoGrande [289], HellaSwag [290], COPA [291], WSC [292], CSQA [293], SIQA [294], C3 [295], CLUEWSC2020 [246], CLUEWSC [246], CLUEWSC-FC [247], ReCoRD [296] Reading Comprehension SQuAD [297], BoolQ [298], SQUADv2 [299], DROP [300], RTE [301], WebQA [302], CMRC2017 [303], CMRC20”
• Subword-augmented Embedding for Cloze Reading Comprehension (2018, methodology): “1 Dataset and Settings To verify the effectiveness of our proposed model, we conduct multiple experiments on three Chinese Machine Reading Comprehension datasets, namely CMRC-2017 (Cui et al., 2017b), People’s Daily (PD) and Children Fairy Tales (CFT) (Cui et al.”