Consensus Attention-based Neural Networks for Chinese Reading Comprehension

By Yiming Cui, Ting-Wei Liu, Z. Chen, Shijin Wang, Guoping Hu. 2016

88 citations recorded by the source corpus.

Reading comprehension has embraced a booming in recent NLP research. Several institutes have released the Cloze-style reading comprehension data, and these have greatly accelerated the research of machine comprehension. In this work, we firstly present Chinese reading comprehension datasets, which consist of People Daily news dataset and Children’s Fairy Tale (CFT) dataset. Also, we propose a consensus attention-based neural network architecture to tackle the Cloze-style reading comprehension problem, which aims to induce a consensus attention over every words in the query. Experimental results show that the proposed neural network significantly outperforms the state-of-the-art baselines in several public datasets. Furthermore, we setup a baseline for Chinese reading comprehension task, and hopefully this would speed up the process for future research.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

PanGu-α: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation (2021, methodology): “For the cloze tasks such as WPLC, PD&CFT, and CMRC2017, the prompts are the context before the positions to be predicted.”
Machine Comprehension Using Match-LSTM and Answer Pointer (2016, methodology): “…of applying neural network models in NLP, there has been much interest in building end-to-end neural architectures for various NLP tasks, including several pieces of work on machine comprehension (Hermann et al., 2015; Hill et al., 2016; Yin et al., 2016; Kadlec et al., 2016; Cui et al., 2016).”
Attention-over-Attention Neural Networks for Reading Comprehension (2016, background): “Liu et al. (2016) proposed to exploit reading comprehension models to other tasks.”
DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applications (2017, methodology): “In detail, cloze-style MRC (Hermann et al., 2015; Hill et al., 2015; Cui et al., 2016) simpliﬁes the task into word prediction on hole-digging synthesis data.”
End-to-End Answer Chunk Extraction and Ranking for Reading Comprehension (2016, background): “…et al. 2015; Kadlec et al. 2016; Trischler et al. 2016b; Dhingra et al. 2016; Chen, Bolton, and Manning 2016; Sordoni, Bachman, and Bengio 2016; Cui et al. 2016a); or for selecting the correct answer by ranking a small set of human-provided candidates (Yin, Ebert, and Sch ¨ utze 2016;…”
A Span-Extraction Dataset for Chinese Machine Reading Comprehension (2019, methodology): “…of machine reading comprehension datasets have been proposed, such as cloze-style reading comprehension (Hermann et al., 2015; Hill et al., 2015; Cui et al., 2016), span-extraction reading comprehension (Rajpurkar et al., 2016; Trischler et al., 2016), open-domain reading comprehension (Nguyen…”
Embracing data abundance: BookTest Dataset for Reading Comprehension (2016, methodology): “…on the original CBT data and they can be compared with recent text-comprehension models utilizing this dataset (Hill et al., 2015; Kadlec et al., 2016; Chen et al., 2016; Sordoni et al., 2016; Dhingra et al., 2016; Trischler et al., 2016b; Weis-senborn, 2016; Cui et al., 2016b; Cui et al., 2016a).”
ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation (2021, unknown): “: PD&CFT [81], CMRC2017 [82], CMRC2019 [83], CHID [84], CHID-FC [68], WPLC [85].”
Subword-augmented Embedding for Cloze Reading Comprehension (2018, methodology): “er z 79.1 75.0 75.3 72.0 GA Reader z 76.8 72.5 73.1 69.6 SAW Reader 78.5 74.9 75.0 71.6 Table 5: Accuracy on CBT dataset. Results marked with zare of previously published works (Dhingra et al., 2017; Cui et al., 2016; Yang et al., 2017). 0 1k 2k 5k 10k 20k Merging Times 76.0 76.5 77.0 77.5 78.0 78.5”
Effective Subword Segmentation for Text Comprehension (2018, background): “RESULTS OF AS READER AND CAS READER ARE FROM [11].”
Probing Prior Knowledge Needed in Challenging Chinese Machine Reading Comprehension (2019, methodology): “e never been studied before. 2.2 Chinese Machine Reading Comprehension and Non-Extractive Question Answering We have seen the construction of spanstyle Chinese machine reading comprehension datasets (Cui et al., 2016, 2018b,a; Shao et al., 2018), using Chinese news reports, books, and Wikipedia arti”
BiPaR: A Bilingual Parallel Dataset for Multilingual and Cross-lingual Reading Comprehension on Novels (2019, background): “, 2015) HLF-RC (Cui et al., 2016) SQuAD (Rajpurkar et al.”
Effective Character-Augmented Word Embedding for Machine Reading Comprehension (2018, methodology): “There are two main kinds of MRC, user-query types [13,24] and cloze-style [7,10,11].”
Dataset for the First Evaluation on Chinese Machine Reading Comprehension (2018, methodology): “Following Cui et al. (Cui et al., 2016), we also use similar way to generate our training data automatically.”
A Sentence Cloze Dataset for Chinese Machine Reading Comprehension (2020, methodology): “PD&CFT (Cui et al., 2016) News, Story Cloze Word 28K 100K WebQA (Li et al.”
From Particular to General: A Preliminary Case Study of Transfer Learning in Reading Comprehension (2016, methodology): “Our results show that even though the AS Reader trained on BookTest outperforms all published models [9, 23, 6, 25, 26, 5, 4] trained on the CBT it fails to solve any of the bAbI tasks with better than 65% accuracy.”