Hierarchical Graph Network for Multi-hop Question Answering

By Yuwei Fang, Siqi Sun, Zhe Gan, R. Radhakrishna Pillai, Shuohang Wang, Jingjing Liu. 2019

198 citations recorded by the source corpus.

In this paper, we present Hierarchical Graph Network (HGN) for multi-hop question answering. To aggregate clues from scattered texts across multiple paragraphs, a hierarchical graph is created by constructing nodes from different levels of granularity (questions, paragraphs, sentences, and entities), the representations of which are initialized with RoBERTa-based context encoders. Given this hierarchical graph, the initial node representations are updated through graph propagation, and multi-hop reasoning is performed via traversing through the graph edges for each subsequent sub-task (e.g., paragraph selection, supporting facts extraction, answer prediction). By weaving heterogeneous nodes into an integral unified graph, this characteristic hierarchical differentiation of node granularity enables HGN to support different question answering sub-tasks simultaneously. Experiments on the HotpotQA benchmark demonstrate that the proposed model achieves new state of the art in both the Distractor and Fullwiki settings.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Reasoning Like Program Executors (2022, methodology): “(ii) Similarly, on HotpotQA (Distractor), specialized model baselines include DFGN (Qiu et al., 2019), SAE (Tu et al., 2020), C2F Reader (Shao et al., 2020) and the SOTA model HGN (Fang et al., 2020).”
Big Bird: Transformers for Longer Sequences (2020, methodology): “Model HotpotQA NaturalQ TriviaQA WikiHop Ans Sup Joint LA SA Full Verified MCQ HGN [26] 82.”
Longformer: The Long-Document Transformer (2020, methodology): “(Glaß et al., 2019) (not GNN) 79.4 86.2 - HGN (Fang et al., 2019) 81.0 87.9 73.0 C2F Reader (Shao et al., 2020) - - 73.9 Table 10: HotpotQA results, distractor setting on the dev set.”
Unsupervised Question Decomposition for Question Answering (2020, methodology): “, 2020) and HGN (Fang et al., 2019), whose methods may be combined with ours.”
Text Modular Networks: Learning to Decompose Tasks in the Language of Existing Models (2021, methodology): “While many multi-hop QA models exist for HotpotQA and DROP, these are often equally complex models (Tu et al., 2020; Fang et al., 2020; Ran et al., 2019) focusing on just one of these datasets.”
Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval (2020, methodology): “In particular, Graph Recurrent Retriever trains a graph traversal model for chain retrieval; TransformerXH (Zhao et al., 2020) and HGN (Fang et al., 2019) explicitly encode the hyperlink graph structure within their answer prediction models.”
Differentiable Reasoning over a Virtual Knowledge Base (2020, methodology): “Even with only 50% edges retained, we see good performance—better than PullNet and the state-of-the-art for a KB-only method (in italics ).”
Multi-hop Question Answering via Reasoning Chains (2019, neutral): “DecompRC (Min et al., 2019b) 55.20 69.63 N QFE (Nishida et al., 2019) 53.86 68.06 Y DFGN (Qiu et al., 2019) 56.31 69.69 Y HGN (Fang et al., 2019) 66.07 79.36 Y SAE (Tu et al., 2019a) 66”
Graph Sequential Network for Reasoning over Sequences (2020, background): “Both tasks require the model to have reasoning ability, and top performance has been achieved with early summarization followed by GNN (Zhou et al., 2019; Xiao et al., 2019; Fang et al., 2019; Tu et al., 2019a).”
Answer Complex Questions: Path Ranker Is All You Need (2021, background): “HGN-reader[7] describes the loss functions of each task, and will not be repeated for simplicity.”
Is Graph Structure Necessary for Multi-hop Question Answering? (2020, unknown): “…et al., 2018) 10.83 40.16 QFE (Nishida et al., 2019) 34.63 59.61 DFGN (Qiu et al., 2019) 33.62 59.82 TAP2 (Glass et al., 2019) 39.77 69.12 HGN (Fang et al., 2019) 43.57 71.03 SAE (Tu et al., 2019a) 45 eral extra modules in the graph fusion block, including query-entity attention, query update…”
Logic-Driven Context Extension and Data Augmentation for Logical Reasoning of Text (2021, methodology): “, 2020) are usually employed for converting texts to logical forms, and graph neural networks (Fang et al., 2019; Huang et al., 2021) and neural module networks (Gupta et al.”
F1 Is Not Enough! Models and Evaluation towards User-Centered Explainable Question Answering (2020, background): “Other approaches are based on question decomposition (Min et al., 2019; Perez et al., 2020), graph/hierarchical structures (Tu et al., 2019; Fang et al., 2019; Asai et al., 2020), virtual knowledge bases (Dhingra et al., 2020) or transformer models (Zhao et al., 2020).”
Case-Based Abductive Natural Language Inference (2020, background): “such as open-domain question answering (Xiong et al., 2021; Fang et al., 2020; Yang et al., 2018), increasing attention is being dedicated to the abstractive setting, where the models are required to compose long chains of facts expressing abstract commonsense and scientific knowledge (Jhamtani and ”
ReasonBERT: Pre-trained to Reason with Distant Supervision (2021, methodology): “model, we also show results for HGN (Fang et al., 2020), which is one of the top ranked models on the HotpotQA leaderboard that uses a more sophisticated model design.”
Answering Any-hop Open-domain Questions with Iterative Document Reranking (2020, methodology): “Following the previous state-of-the-art method [10], we use ALBERT-xxlarge [16] as the pre-trained language model.”
Data-QuestEval: A Referenceless Metric for Data-to-Text Semantic Evaluation (2021, methodology): “To learn representations from structured data, several approaches have been proposed – e.g. a hierarchical network (Fang et al., 2019).”