Multi-Granularity Hierarchical Attention Fusion Networks for Reading Comprehension and Question Answering

By Wei Wang, Chen Wu, Ming Yan. 2018

178 citations recorded by the source corpus.

This paper describes a novel hierarchical attention network for reading comprehension style question answering, which aims to answer questions for a given narrative paragraph. In the proposed method, attention and fusion are conducted horizontally and vertically across layers at different levels of granularity between question and paragraph. Specifically, it first encode the question and paragraph with fine-grained language embeddings, to better capture the respective representations at semantic level. Then it proposes a multi-granularity fusion approach to fully fuse information from both global and attended representations. Finally, it introduces a hierarchical attention network to focuses on the answer span progressively with multi-level soft-alignment. Extensive experiments on the large-scale SQuAD, TriviaQA dataset validate the effectiveness of the proposed method. At the time of writing the paper, our model achieves state-of-the-art on the both SQuAD and TriviaQA Wiki leaderboard as well as two adversarial SQuAD datasets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

U-Net: Machine Reading Comprehension with Unanswerable Questions (2018, background): “…Currently, end-to-end neural network models have achieved great successes for machine reading comprehension (Seo et al. 2016; Kumar et al. 2015; Sukhbaatar et al. 2015; Cui et al. 2016; Xiong, Zhong, and Socher 2016; Dhingra et al. 2016; Shen et al. 2016; Hu et al. 2017; Wang, Yan, and Wu 2018).”
Multi-step Retriever-Reader Interaction for Scalable Open-domain Question Answering (2019, methodology): “Open-domain QA systems such as R 3 (Wang et al., 2018a) and DS-QA (Lin et al., 2018) have sophisticated retriever models where the reader and retriever are jointly trained.”
ERNIE 2.0: A Continual Pre-training Framework for Language Understanding (2019, background): “• QNLI: Question Natural Language Inference (QNLI)[25, 26] is a corpus that would tell whether the relationship between a pair of given texts are question-answer.”
Cognitive Graph for Multi-Hop Reading Comprehension at Scale (2019, methodology): “Recently, BERT (De-vlin et al., 2018) has become one of the most successful language representation models on various NLP tasks, including SQuAD (Rajpurkar et al., 2016).”
MultiQA: An Empirical Investigation of Generalization and Transfer in Reading Comprehension (2019, background): “M ULTI QA obtained 66.7 EM and 73.1 F 1 , outperforming BERT-large (60.4EM, 66.3F1) by a signiﬁcant margin, and also substantially improving state-of-the-art re-sults (56.0 EM, 61.3 F 1 , (Hu et al., 2018) and 52.1 EM, 62.7 F 1 , (Wang et al., 2018)).”
A Span-Extraction Dataset for Chinese Machine Reading Comprehension (2019, background): “…dataset, various neural network approaches have been proposed and made a big advancement in this area (Kadlec et al., 2016; Cui et al., 2017; Dhingra et al., 2017; Wang and Jiang, 2016; Xiong et al., 2016; Liu et al., 2017; Wang et al., 2017; Hu et al., 2018; Wang et al., 2018; Yu et al., 2018).”
Improving Machine Reading Comprehension with General Reading Strategies (2018, methodology): “Given informative factoid questions, stateof-the-art attention-based neural models (Wang et al., 2018b; Hu et al., 2018) designed for extractive MRC tasks have already achieved very high performance based on the local context.”
Read + Verify: Machine Reading Comprehension with Unanswerable Questions (2018, methodology): “.0 86.9 89.5 Table 1: Comparison of different approaches on the SQuAD 2.0 test set, extracted on Aug 28, 2018: Levy et al. (2017)1, Clark et al. (2018)2, Liu et al. (2018b)3, Huang et al. (2018)4 and Wang et al. (2018)5. yindicates unpublished works. Implementation We use the Reinforced Mnemonic Rea”
Towards Explainable NLP: A Generative Explanation Framework for Text Classification (2018, background): “Deep learning methods have produced state-ofthe-art results in many natural language processing (NLP) tasks (Vaswani et al., 2017; Yin et al., 2018; Peters et al., 2018; Wang et al., 2018; Hancock et al., 2018; Ma et al., 2018).”
From 'F' to 'A' on the N.Y. Regents Science Exams: An Overview of the Aristo Project (2019, background): “, 2010), and now with high-performing neural systems that can answer short questions provided they are given a text that contains the answer (e.g., Seo et al., 2016; Wang et al., 2018).”
Learning Cross-Context Entity Representations from Text (2020, background): “We deﬁne a compatibility score between the entity e and the context x as the scaled cosine similarity where the scaling factor a is a learned parameter, following Wang et al. (2018a).”
Question Answering Using Hierarchical Attention on Top of BERT Features (2019, methodology): “To allow for more freedom of the aligning process, we used a bilinear self-alignment attention function on the passage representation, introduced in (Wang et al., 2018).”
Addressing Semantic Drift in Generative Question Answering with Auxiliary Extraction (2021, background): “Current mainstream approaches (Chen et al., 2017; Wang et al., 2018; Yan et al., 2018) treat MRC as a process of extracting a consecutive piece of text from a document to a given question.”
I Know What You Want: Semantic Learning for Text Comprehension (2018, background): “Wang et al. 2018; Rajpurkar, Jia, and Liang 2018), which constrain answers to all possible text spans within the reference document, researchers are investigating the models with more logical reasoning and content understanding (Wang et al. 2018; Wang, Yan, and Wu 2018).”
Alignment over Heterogeneous Embeddings for Question Answering (2019, unknown): “Wang and Jiang, 2016; Wang et al., 2017b; Ty-moshenko et al., 2017; Xiong et al., 2016a; Wang et al., 2018; Radford et al., 2018; Li et al., 2018, inter alia).”
StructuralLM: Structural Pre-training for Form Understanding (2021, methodology): “For the document visual question answering task, we treat it as an extractive QA task and build a token-level classiﬁer on the top of token representations, which is usually used in Machine Reading Comprehension (MRC) (Rajpurkar et al., 2016; Wang et al., 2018).”
CogLTX: Applying BERT to Long Texts (2020, background): “Previous works [49, 33] tried to aggregate results from each window by mean-pooling, max-pooling, or an additional MLP or LSTM over them; but these methods are still weak at long-distance interaction and need O(512(2) · L/512) = O(512L) space, which in practice is still too large to train a BERT-lar”
Cross-Lingual Transfer Learning for Question Answering (2019, background): “Question answering (QA) has drawn much attention in the past few years, and end-to-end deep learning has demonstrated promising performance on QA (Yu et al., 2018; Xiong et al., 2016; Seo et al., 2016; Huang et al., 2017; Wang et al., 2018; Liu et al., 2017; Hu et al., 2017; Wang et al., 2017).”