Hierarchical Attention Flow for Multiple-Choice Reading Comprehension

By Haichao Zhu, Furu Wei, Bing Qin, Ting-Wei Liu. 2018

77 citations recorded by the source corpus.

In this paper, we focus on multiple-choice reading comprehension which aims to answer a question given a passage and multiple candidate options. We present the hierarchical attention flow to adequately leverage candidate options to model the interactions among passages, questions and candidate options. We observe that leveraging candidate options to boost evidence gathering from the passages play a vital role in this task, which is ignored in previous works. In addition, we explicitly model the option correlations with attention mechanism to obtain better option representations, which are further fed into a bilinear layer to obtain the ranking score for each option. On a large-scale multiple-choice reading comprehension dataset (i.e. the RACE dataset), the proposed model outperforms two previous neural network baselines on both RACE-M and RACE-H subsets and yields the state-of-the-art overall results.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Option Comparison Network for Multiple-choice Reading Comprehension (2019, background): “…models have been proposed for MCRC and achieve promising re-sults in recent years (Chen et al., 2016; Yin et al., 2016; Trischler et al., 2016; Dhingra et al., 2017; Tay et al., 2018; Parikh et al., 2018; Zhu et al., 2018; Wang et al., 2018; Xu et al., 2017; Sun et al., 2018; Zhang et al., 2019).”
Dual Co-Matching Network for Multi-choice Reading Comprehension (2019, methodology): “e machine reading comprehension datasets. #o is the average number of candidate options for each question. #p and #q are the number of documents and questions in the dataset. Model RACE-M/H RACE HAF (Zhu et al. 2018) 45.0/46.4 46.0 MRU (Tay, Tuan, and Hui 2018) 57.7/47.4 50.4 HCM (Wang et al. 2018b)”
Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning (2019, background): “Multiway attention (Wang et al., 2018a; Zhu et al., 2018) has been shown to be effective in capturing the interactions between each pair of input para-graph, question and candidate answers, leading to better context interpretation, while BERT ﬁne-tuning (Devlin et al., 2018) also shows its…”
A Co-Matching Model for Multi-choice Reading Comprehension (2018, methodology): “• HAF (Zhou et al., 2018) considers not only the matching between the three sequences, namely, passage, question and candidate answer, but also the matching between the candidate answers.”
Improving Machine Reading Comprehension with General Reading Strategies (2018, methodology): “, 2018), applying hierarchical attention strategies (Zhu et al., 2018; Wang et al., 2018a), using reinforcement learning to determine the choice of multiple attention strategies (Xu et al.”
How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence (2020, methodology): “We implemented several representative question answering models, including BiDAF (Seo et al., 2016), BERT (Devlin et al., 2019), Co-matching (Wang et al., 2018), and HAF (Zhu et al., 2018).”
FCM: A Fine-grained Comparison Model for Multi-turn Dialogue Reasoning (2021, unknown): “For the multi-choice reasoning task (Zhu et al., 2018), there are only slight differences between the illogical response and the dialogue history, so a fine-grained comparison mechanism is required to infer the logic between each candidate response and the dialogue history.”
Incorporating Commonsense Knowledge Graph in Pretrained Models for Social Commonsense Tasks (2020, background): “KG Attentive Representations Motivated by previous work on question answering (Seo et al., 2017; Zhu et al., 2018; Wang et al., 2018; Huang et al., 2019), which uses attention among different segments of the input, here we treat the knowledge tuples as a new segment.”
Incorporating Structured Commonsense Knowledge in Story Completion (2018, methodology): “Usually they are solved by match-based model like QANET (Yu et al. 2018), hierarchical attention model like HAF (Zhu et al. 2018), and dynamic fusion based model like DFN (Xu et al. 2018).”
Convolutional Spatial Attention Model for Reading Comprehension with Multiple-Choice Questions (2018, methodology): “Also, when compared to the Hierarchical Attention Flow model (Zhu et al. 2018), our CSA model show substantial improvements, indicating that utilizing the attentions in each hierarchy and using convolutional neural network to extract the most representative features from the spatial attentions are u”