Cross-Sentence N-ary Relation Extraction with Graph LSTMs

By Nanyun Peng, Hoifung Poon, Chris Quirk, Kristina Toutanova, Wen-tau Yih. 2017

538 citations recorded by the source corpus.

Past work in relation extraction has focused on binary relations in single sentences. Recent NLP inroads in high-value domains have sparked interest in the more general setting of extracting n-ary relations that span multiple sentences. In this paper, we explore a general relation extraction framework based on graph long short-term memory networks (graph LSTMs) that can be easily extended to cross-sentence n-ary relation extraction. The graph formulation provides a unified way of exploring different LSTM approaches and incorporating various intra-sentential and inter-sentential dependencies, such as sequential, syntactic, and discourse relations. A robust contextual representation is learned for the entities, which serves as input to the relation classifier. This simplifies handling of relations with arbitrary arity, and enables multi-task learning with related relations. We evaluate this framework in two important precision medicine settings, demonstrating its effectiveness with both conventional supervised learning and distant supervision. Cross-sentence extraction produced larger knowledge bases. and multi-task learning significantly improved extraction accuracy. A thorough analysis of various LSTM approaches yielded useful insight the impact of linguistic analysis on extraction accuracy.

Subfields: Artificial Intelligence, Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

Citations in the corpus

Sentence-State LSTM for Text Representation (2018, background): “The closest previous work in NLP includes the use of convolutional neural networks (Bastings et al., 2017; Marcheggiani and Titov, 2017) and DAG LSTMs (Peng et al., 2017) for modelling syntactic structures.”
Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction (2018, methodology): “Among those studies, many papers assume the entity boundaries are given, such as (Clark and Manning, 2016), Adel and Sch¨utze (2017) and Peng et al. (2017).”
Novel positional encodings to enable tree-based transformers (2019, methodology): “With a few changes, these models can be extended to cover graph-like structures as well (Peng et al., 2017).”
Attention Guided Graph Convolutional Networks for Relation Extraction (2019, methodology): “For the cross-sentence n-ary relation extraction task, we use the dataset introduced in (Peng et al., 2017), which contains 6,987 ternary relation instances and 6,087 binary relation instances extracted from PubMed.”
Document-Level Relation Extraction with Adaptive Thresholding and Localized Context Pooling (2020, methodology): “Later work extends the idea by improving neural architectures (Peng et al. 2017; Verga, Strubell, and McCallum 2018; Song et al. 2018; Jia, Wong, and Poon 2019; Gupta et al. 2019) or adding more types of edges (Christopoulou, Miwa, and Ananiadou 2019; Nan et al.”
Question Answering by Reasoning Across Documents with Graph Convolutional Networks (2018, background): “Graph neural networks have been shown successful on a number of NLP tasks (Marcheggiani and Titov, 2017; Bastings et al., 2017; Zhang et al., 2018a), including those involving document level modeling (Peng et al., 2017).”
A general framework for information extraction using dynamic span graphs (2019, methodology): “…explored joint modeling (Miwa and Bansal, 2016; Zhang et al., 2017; Singh et al., 2013; Yang and Mitchell, 2016)) and multi-task learning (Peng and Dredze, 2015; Peng et al., 2017; Luan et al., 2018a, 2017a) as methods to share representational strength across related information extraction tasks.”
GraphRel: Modeling Text as Relational Graphs for Joint Entity and Relation Extraction (2019, result): “There are also some works (Peng et al., 2017; Zhang et al., 2018; Qian et al.,”
Graph Neural Networks and Their Current Applications in Bioinformatics (2021, methodology): “…the rapid processing of massive data, such as predicting the properties of chemical molecules (Duvenaud et al., 2015), extracting text relationship (Peng et al., 2017), reasoning the structure of graphics and images (Wang et al., 2018), link prediction and node clustering of social networks (Zhang…”
Improving Graph Neural Network Representations of Logical Formulae with Subgraph Pooling (2019, background): “In prior work [13], [23], there would be matrices for each edge type, (e.g., instead of just U i , there would be U ki for each edge type k ).”
Relation Extraction Using Distant Supervision (2018, background): “N-ary relation extraction also remains an active research area [15, 46].”
Reasoning with Latent Structure Refinement for Document-Level Relation Extraction (2020, background): “Unlike existing document-level datasets (Li et al., 2016a; Quirk and Poon, 2017; Peng et al., 2017; Verga et al., 2018; Jia et al., 2019)”
Neural Models for Reasoning over Multiple Mentions Using Coreference (2018, methodology): “Our resulting layer is structurally similar to GraphLSTMs (Peng et al., 2017), with an additional attention mechanism over the graph edges.”
From POS tagging to dependency parsing for biomedical event extraction (2018, methodology): “hat are standardly used for development and evaluation of syntactic processing tools. Recent work on biomedical relation extraction has highlighted the particular importance of syntactic information (Peng et al., 2017). Despite this, that work, and most other related work, has simply adopted a tool ”
More Data, More Relations, More Context and More Openness: A Review and Outlook for Relation Extraction (2020, background): “Quirk and Poon (2017); Peng et al. (2017) build datasets by DS. Li et al. (2016); Peng et al. (2017) propose datasets for speciﬁc domains.”
Chinese NER Using Lattice LSTM (2018, methodology): “, 2017), dependencydiscourse DAGs (Peng et al., 2017), as well as speech tokenization lattice (Sperber et al.”
Entity, Relation, and Event Extraction with Contextualized Span Representations (2019, background): “…to structured prediction models (Li et al., 2013; Yang and Mitchell, 2016; Li and Ji, 2014) and neural scoring functions (Nguyen and Nguyen, 2019), or as a guide for the construction of neural architectures (Peng et al., 2017; Zhang et al., 2018; Sha et al., 2018; Christopoulou et al., 2018).”
DocRED: A Large-Scale Document-Level Relation Extraction Dataset (2019, background): “However, these datasets are either constructed via distant supervision (Quirk and Poon, 2017; Peng et al., 2017) with inevitable wrong labeling problem, or limited in specific domain (Li et al., 2016; Peng et al., 2017).”