Is Graph Structure Necessary for Multi-hop Reasoning?
By Nan Shao, Yiming Cui, Ting-Wei Liu, Shijin Wang, Guoping Hu. 2020
17 citations recorded by the source corpus.
Recently, many works attempt to model texts as graph structure and introduce graph neural networks to deal with it on many NLP this http URL this paper, we investigate whether graph structure is necessary for multi-hop reasoning tasks and what role it plays. Our analysis is centered on HotpotQA. We use the state-of-the-art published model, Dynamically Fused Graph Network (DFGN), as our baseline. By directly modifying the pre-trained model, our baseline model gains a large improvement and significantly surpass both published and unpublished works. Ablation experiments established that, with the proper use of pre-trained models, graph structure may not be necessary for multi-hop reasoning. We point out that both the graph structure and the adjacency matrix are task-related prior knowledge, and graph-attention can be considered as a special case of self-attention. Experiments demonstrate that graph-attention or the entire graph structure can be replaced by self-attention or Transformers, and achieve similar results to the previous state-of-the-art model achieved.
Subfields: Natural Language Processing, Graph Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Natural Language Processing
- Iz Beltagy
- Zhe Gan
- Arman Cohan
- Jingjing Liu
- Heike Adel
- Shuohang Wang
- Matthew E. Peters
- R. Radhakrishna Pillai
In Graph Learning
Citations in the corpus
- Longformer: The Long-Document Transformer (2020, methodology): “All the better performing models for HotpotQA (Shao et al., 2020; Fang et al., 2019) use multi-stage approaches plus GNNs or graph network of entities, which seem to encode an important inductive bias for the task.12 Furthermore, the 10 paragraphs come from 10 different documents and the concatenate”
- Hierarchical Graph Network for Multi-hop Question Answering (2019, methodology): “C2F Reader (Shao et al., 2020) uses graph attention or self-attention on entity graph, and argues that this graph may not be necessary for multi-hop reasoning.”
- F1 Is Not Enough! Models and Evaluation towards User-Centered Explainable Question Answering (2020, methodology): “Previous work on XQA mostly focuses on developing models that predict the correct answer and, independent of this, the correct explanation (Yang et al., 2018; Qi et al., 2019; Shao et al., 2020).”