Relation Extraction: Perspective from Convolutional Neural Networks

By Thien Huu Nguyen, Ralph Grishman. VS@HLT-NAACL, 2015

509 citations recorded by the source corpus.

Up to now, relation extraction systems have made extensive use of features generated by linguistic analysis modules. Errors in these features lead to errors of relation detection and classification. In this work, we depart from these traditional approaches with complicated feature engineering by introducing a convolutional neural network for relation extraction that automatically learns features from sentences and minimizes the dependence on external toolkits and resources. Our model takes advantages of multiple window sizes for filters and pre-trained word embeddings as an initializer on a non-static architecture to improve the performance. We emphasize the relation extraction problem with an unbalanced corpus. The experimental results show that our system significantly outperforms not only the best baseline systems for relation extraction but also the state-of-the-art systems for relation classification.

Subfields: Artificial Intelligence, Information Systems

View Paper: https://www.semanticscholar.org/paper/eb9fb8385c5824b029633c0cb68a8fb8573380ad

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

Citations in the corpus

Attention Guided Graph Convolutional Networks for Relation Extraction (2019, background): “Recently, sequencebased models leverages different neural networks to extract relations, including convolutional neural networks (Zeng et al., 2014; Nguyen and Grishman, 2015; Wang et al., 2016), recurrent neural networks (Zhou et al.”
Chemical-induced disease relation extraction via convolutional neural network (2017, methodology): “Nguyen and Grishman (27) employed a CNN-based model utilizing multiple sizes of filters to conduct the relation classification task, and they achieved an F-score of 82.”
Position-aware Attention and Supervised Data Improve Slot Filling (2017, background): “This can be explained by noting that the ﬁlters in CNNs are essentially a form of “fuzzy n-gram patterns”.”
Neural Temporal Relation Extraction (2017, methodology): “These settings are identical or similar to those used in neural sentence modeling work (Nguyen and Grishman, 2015; Zhang and Wallace, 2015; Kim, 2014) and were validated on the SemEval development set.”
A Comprehensive Survey on Automatic Knowledge Graph Construction (2023, methodology): “MIML CNN [194] uses a CNN to proceed with each sentence bag, then leverages a cross-sentence pooling operation to derive an entity-pair representation for multi-label relation modeling.”
Refining Sample Embeddings with Relation Prototypes to Enhance Continual Relation Extraction (2021, background): “…and utilizing various deep neural networks to discover the relations be-tween entities given contexts, including: (1) Convolutional neural networks (CNNs) (Zeng et al., 2014, 2015; Nguyen and Grishman, 2015; Lin et al., 2016; Ji et al., 2017) can effectively extract local textual features.”
Relation Extraction Using Distant Supervision (2018, background): “Supervised approaches ([43], [41], [72]) need annotated training data which is expensive to produce.”
Learning Domain-Specific Word Embeddings from Sparse Cybersecurity Texts (2017, background): “…as Named Entity Recognition (Lample et al. 2016), (Santos and Guimaraes 2015), syntactic parsing (Bansal, Gimpel, and Livescu 2014), semantic relation extraction (Nguyen and Grishman 2015), (Fu et al. 2014) and sentiment analysis (Maas et al. 2011), (Tang et al. 2014), to boost their performance.”
A Survey on Knowledge Graphs: Representation, Acquisition, and Applications (2020, methodology): “ConMask [84] proposes relationship-dependent content masking over the entity description to select relevant snippets of given relations, and CNN-based target fusion to complete the knowledge graph with unseen entities.”
Continual Relation Learning via Episodic Memory Activation and Reconsolidation (2020, unknown): “…both supervised RE models (Zelenko et al., 2003; Zhou et al., 2005; Gormley et al., 2015; Socher et al., 2012; Liu et al., 2013; Zeng et al., 2014; Nguyen and Grishman, 2015; dos Santos et al., 2015; Xu et al., 2015; Liu et al., 2015; Miwa and Bansal, 2016) and distantly supervised models…”
Combining Neural Networks and Log-linear Models to Improve Relation Extraction (2015, methodology): “Following the common practice of domain adaptation research on this dataset (Plank and Moschitti, 2013; Nguyen and Grishman, 2014; Nguyen et al., 2015c; Gormley et al., 2015), we use news (the union of bn and nw ) as the training data, a half of bc as the development set and the remainder ( cts ,…”
More Data, More Relations, More Context and More Openness: A Review and Outlook for Relation Extraction (2020, background): “, 2012; Miwa and Bansal, 2016) that learn compositional representations for sentences recursively, convolutional neural networks (CNNs) (Liu et al., 2013; Zeng et al., 2014; Santos et al., 2015; Nguyen and Grishman, 2015b; Zeng et al., 2015; Huang and Wang, 2017) that effectively model local textual”
Identifying Customer Needs from User-Generated Content (2019, background): “In NLP, CNNs demonstrate state-of-the-art performance with minimum tuning in such problems as relation extraction (Nguyen and Grishman, 2015), named entity recognition (Sun et al., 2015) and sentiment analysis (Santos and Gatti, 2014).”
Drug-Drug Interaction Extraction via Recurrent Neural Network with Multiple Attention Layers (2017, methodology): “In recent years, attention mechanism and various neural networks are applied to relation extraction [18]–[22].”
Event Detection and Domain Adaptation with Convolutional Neural Networks (2015, background): “, 2011), relation classification and extraction (Zeng et al., 2014; Nguyen and Grishman, 2015).”
Probing Linguistic Features of Sentence-Level Representations in Relation Extraction (2020, methodology): “(2014) and Nguyen and Grishman (2015), who both use a convolutional neural network for relation extraction.”
Neural Unsupervised Domain Adaptation in NLP—A Survey (2020, result): “This bears some similarity to early work on leveraging general background knowledge (embeddings trained on general data) for domain adaptation (Plank and Moschitti, 2013; Nguyen and Grishman, 2015; Li et al., 2018a), and also relates to recent work on pre-training (Section 5.”
WikiReading: A Novel Large-scale Language Understanding Task over Wikipedia (2016, background): “A growing amount of research in natural language understanding (NLU) explores end-to-end deep neural network (DNN) architectures for tasks such as text classiﬁcation (Zhang et al., 2015), relation extraction (Nguyen and Grishman, 2015), and question answering (Weston et al., 2015).”