Combining Neural Networks and Log-linear Models to Improve Relation Extraction

By Thien Huu Nguyen, Ralph Grishman. 2015

103 citations recorded by the source corpus.

The last decade has witnessed the success of the traditional feature-based method on exploiting the discrete structures such as words or lexical patterns to extract relations from text. Recently, convolutional and recurrent neural networks has provided very effective mechanisms to capture the hidden structures within sentences via continuous representations, thereby significantly advancing the performance of relation extraction. The advantage of convolutional neural networks is their capacity to generalize the consecutive k-grams in the sentences while recurrent neural networks are effective to encode long ranges of sentence context. This paper proposes to combine the traditional feature-based method, the convolutional and recurrent neural networks to simultaneously benefit from their advantages. Our systematic evaluation of different network architectures and combination methods demonstrates the effectiveness of this approach and results in the state-of-the-art performance on the ACE 2005 and SemEval dataset.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Zhiyuan Liu
2. Maosong Sun
3. Xu Han
4. Mo Yu
5. Yankai Lin
6. Jie Zhou
7. Shiyu Chang
8. Tianyu Gao

Citations in the corpus

• Natural Language Processing for EHR-Based Computational Phenotyping (2018, methodology): “Deep learning has been applied on various NLP applications, including semantic representation [146], semantic analysis [147, 148], information retrieval [149, 150], entity recognition [151, 152], relation extraction [153-156], and event detection [157, 158].”
• Rethinking Cooperative Rationalization: Introspective Extraction and Complement Control (2019, methodology): “For the relation classification task, since the model needs to be aware of the two target entities, we add the relative position features following Nguyen and Grishman (2015).”
• More Data, More Relations, More Context and More Openness: A Review and Outlook for Relation Extraction (2020, background): “…Bansal, 2016) that learn compositional representations for sentences recursively, convolutional neural networks (CNNs) (Liu et al., 2013 Wang, 2015; Nguyen and Grishman, 2015a; Vu et al., 2016; Zhang et al., 2015) that can better handle long sequential data, graph neural networks (GNNs) (Zhang et…”
• Extracting Multiple-Relations in One-Pass with Pre-Trained Transformers (2019, methodology): “Existing approaches on MRE tasks (Qu et al., 2014; Gormley et al., 2015; Nguyen and Grishman, 2015) propose to adopt the methods for Single-Relation Extraction tasks to solve MRE tasks.”
• Text feature extraction based on deep learning: a review (2017, background): “Deep learning is by learning a kind of deep nonlinear network structure and implementing complex function approximation, according to the characterization of the input data distributed, and in the case of sample set, the essence characteristic of the data set [63] is seldom studied.”
• Graph Convolutional Networks With Argument-Aware Pooling for Event Detection (2018, methodology): “Specifically, we repeat the model selection procedure in Table 1 of the previous section to select the best network architecture for each pooling method of comparison in {ANCHOR, OVERALL, DYNAMIC} (using the ACE 2005 development dataset).”