Graph Convolution over Pruned Dependency Trees Improves Relation Extraction

By Yuhao Zhang, Peng Qi, Christopher D. Manning. 2018

796 citations recorded by the source corpus.

Dependency trees help relation extraction models capture long-range relations between words. However, existing dependency-based models either neglect crucial information (e.g., negation) by pruning the dependency trees too aggressively, or are computationally inefficient because it is difficult to parallelize over different tree structures. We propose an extension of graph convolutional networks that is tailored for relation extraction, which pools information over arbitrary dependency structures efficiently in parallel. To incorporate relevant information while maximally removing irrelevant content, we further apply a novel pruning strategy to the input trees by keeping words immediately around the shortest path between the two entities among which a relation might hold. The resulting model achieves state-of-the-art performance on the large-scale TACRED dataset, outperforming existing sequence and dependency-based neural models. We also show through detailed analysis that this model has complementary strengths to sequence models, and combining them further improves the state of the art.

Subfields: Artificial Intelligence, Natural Language Processing, Graph Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Graph Learning

Citations in the corpus

Graph-based, Self-Supervised Program Repair from Diagnostic Feedback (2020, methodology): “Several works use GNNs to model the structure of text (Yasunaga et al., 2017; Zhang et al., 2018) and more recently, source code (Allamanis et al.”
ERNIE: Enhanced Language Representation with Informative Entities (2019, methodology): “The results of CNN, PA-LSTM, and C-GCN come from the paper by Zhang et al. (2018), which are the best results of CNN, RNN, and GCN respectively.”
Simplifying Graph Convolutional Networks (2019, methodology): “We replace the GCN in DCGCN (Zhang et al., 2018b) with an SGC and get 71.0% and 76.2% on NCI1 and COLLAB datasets (Yanardag & Vishwanathan, 2015) respectively, which is on par with an GCN counterpart, but far behind GIN.”
K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters (2020, methodology): “This directly demonstrates injecting knowledge individually in K-A DAPTER way would help models make full use of C-GCN (Zhang et al., 2018) 69.90 63.30 66.40 BERT-base (Zhang et al., 2019) 67.23 64.81 66.00 ERNIE (Zhang et al., 2019) 69.97 66.08 67.97 BERT-large (Baldini Soares et al., 2019) - -…”
PTR: Prompt Tuning with Rules for Text Classification (2021, methodology): “We carry out our experiments on four relaction clas-siﬁcation datasets: TACRED (Zhang et al., 2017), TACREV (Alt et al., 2020), ReTACRED (Sto-ica et al., 2021), and SemEval 2010 Task 8 (Se-mEval) (Hendrickx et al., 2010).”
Aligning Cross-Lingual Entities with Multi-Aspect Information (2019, methodology): “Inspired by previous studies (Zhang et al., 2018; Wang et al., 2018), we also adopt GCNs in this work to collect evidence from multilingual KG structures and to learn cross-lingual embeddings of entities.”
Aspect-based sentiment analysis via affective knowledge enhanced graph convolutional networks (2021, neutral): “Recently, graph network-based models have achieve promising performance in many NLP tasks, such as text classification [24, 25], relation extraction [26], Chinese NER [27,28].”
Relational Graph Attention Network for Aspect-based Sentiment Analysis (2020, background): “The idea described above is partially inspired by previous ﬁndings (He et al., 2018a; Zhang et al., 2018; He et al., 2018b) that it could be sufﬁcient to focus on a small subset of context words syntactically close to the target aspect.”
GraphRel: Modeling Text as Relational Graphs for Joint Entity and Relation Extraction (2019, background): “There are also some works (Peng et al., 2017; Zhang et al., 2018; Qian et al., 2019; Luan et al., 2019) about considering dependency structure of word sequence for relation extraction.”
BertGCN: Transductive Text Classification by Combining GNN and BERT (2021, methodology): “…into (Wu et al., 2020): graph convolutional networks (Kipf and Welling, 2016a; Wu et al., 2019), graph attention networks (Veliˇckovi´c et al., 2017; Zhang et al., 2018a), graph auto-encoder (Cao et al., 2016; Kipf and Welling, 2016b), graph generative networks (De Cao and Kipf, 2018; Li et al.,…”
Attention Guided Graph Convolutional Networks for Relation Extraction (2019, methodology): “Following (Zhang et al., 2018), we concate-nate the sentence representation and entity representations to get the ﬁnal representation for classi-ﬁcation.”