End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures

By Makoto Miwa, Mohit Bansal. 2016

1,268 citations recorded by the source corpus.

We present a novel end-to-end neural model to extract entities and relations between them. Our recurrent neural network based model captures both word sequence and dependency tree substructure information by stacking bidirectional tree-structured LSTM-RNNs on bidirectional sequential LSTM-RNNs. This allows our model to jointly represent both entities and relations with shared parameters in a single model. We further encourage detection of entities during training and use of entity information in relation extraction via entity pretraining and scheduled sampling. Our model improves over the state-of-the-art feature-based model on end-to-end relation extraction, achieving 12.1% and 5.7% relative error reductions in F1-score on ACE2005 and ACE2004, respectively. We also show that our LSTM-RNN based model compares favorably to the state-of-the-art CNN based model (in F1-score) on nominal relation classification (SemEval-2010 Task 8). Finally, we present an extensive ablation analysis of several model components.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Zero-Shot Relation Extraction via Reading Comprehension (2017, methodology): “Lastly, we retrain an off-the-shelf relation extraction system (Miwa and Bansal, 2016), which has shown promising results on a number of benchmarks.”
A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks (2016, methodology): “02 Søgaard and Goldberg (2016) 95.”
SciTaiL: A Textual Entailment Dataset from Science Question Answering (2018, methodology): “Also, relation extraction methods (Miwa and Bansal 2016) commonly use the output embeddings from an LSTMs (Hochreiter and Schmidhuber 1997) to calculate context-dependent representation of the entities.”
Continual Few-shot Relation Learning via Embedding Space Regularization and Data Augmentation (2022, methodology): “Traditionally, RE methods were built by considering a fixed static set of relations (Miwa and Bansal, 2016; Han et al., 2018a).”
Curriculum-Meta Learning for Order-Robust Continual Relation Extraction (2021, methodology): “The conventional relation extraction methods could be categorized into three domains by the way data is used: supervised methods (Zelenko, Aone, and Richardella 2002; Liu et al. 2013; Zeng et al. 2014; Lin et al. 2016; Miwa and Bansal 2016), semi-supervised methods (Chen et al.”
Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling (2017, background): “, 2015), including stacking them on top of a sequential BiLSTM (Miwa and Bansal, 2016).”
Continual Relation Learning via Episodic Memory Activation and Reconsolidation (2020, background): “…Socher et al., 2012; Liu et al., 2013; Zeng et al., 2014; Nguyen and Grishman, 2015; dos Santos et al., 2015; Xu et al., 2015; Liu et al., 2015; Miwa and Bansal, 2016) and distantly supervised models (Bunescu and Mooney, 2007; Mintz et al., 2009; Riedel et al., 2010; Hoffmann et al., 2011;…”
Graph Convolution over Pruned Dependency Trees Improves Relation Extraction (2018, background): “One common approach to leverage dependency information is to perform bottom-up or top-down computation along the parse tree or the subtree below the lowest common ancestor (LCA) of the entities (Miwa and Bansal, 2016).”
Neural Snowball for Few-Shot Relation Learning (2019, background): “Besides, dependency parsing trees have also been proved to be efﬁcient in RE (Xu et al., 2015; Liu et al., 2015; Miwa and Bansal, 2016).”
Assessing The Factual Accuracy of Generated Text (2019, methodology): “Miwa and Bansal [18] has a two-phase model that first extracts entity candidates and then predicts relations based on the parsed tree-structure of the sentence.”
A Hierarchical Multi-task Approach for Learning Embeddings from Semantic Tasks (2018, methodology): “To be able to compare our results with previous works (Bekoulis et al. 2018; Miwa and Bansal 2016; Katiyar and Cardie 2017) on EMD, we identify the head of the entity mention rather than the whole mention.”
A general framework for information extraction using dynamic span graphs (2019, methodology): “work has focused on coupling different IE tasks as in joint modeling of entities and relations (Miwa and Bansal, 2016; Zhang et al., 2017), entities and coreferences (Hajishirzi et al.”
Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey (2021, background): “(2021) study structure-linearization for joint entity and relation extraction (Li et al., 2014; Miwa and Bansal, 2016).”
GraphRel: Modeling Text as Relational Graphs for Joint Entity and Relation Extraction (2019, methodology): “The BiLSTM-GCN encoder part of our model re-sembles the BiLSTM-TreeLSTM model proposed by Miwa and Bansal (2016), as they also stack a dependency tree on top of sequences to jointly model entities and relations.”
LISA: Explaining Recurrent Neural Network Judgments via Layer-wIse Semantic Accumulation and Example to Pattern Transformation (2018, background): “, 2016; Ma and Hovy, 2016), relation extraction (Vu et al., 2016a; Miwa and Bansal, 2016; Gupta et al., 2016, 2018c), language modeling (Mikolov et al.”
Incorporating Relation Paths in Neural Relation Extraction (2016, methodology): “(Socher et al., 2012) uses a recursive neural network in relation extraction, and (Xu et al., 2015; Miwa and Bansal, 2016) further use LSTM. (Zeng et al., 2014; dos Santos et al., 2015) adopt CNN in this task, and (Zeng et al., 2015; Lin et al., 2016) combine attention-based multiinstance learning…”
Entity, Relation, and Event Extraction with Contextualized Span Representations (2019, methodology): “For named entity and relation extraction we follow the train / dev / test split from Miwa and Bansal (2016).”
Document-Level Relation Extraction with Adaptive Thresholding and Localized Context Pooling (2020, methodology): “Existing work mainly focuses on sentence-level relation extraction, i.e., predicting the relationship between entities in a single sentence (Zeng et al., 2014; Miwa and Bansal, 2016; Zhang et al., 2018).”