Neural Relation Extraction with Selective Attention over Instances

By Yankai Lin, Shiqi Shen, Zhiyuan Liu, Huanbo Luan, Maosong Sun. 2016

1,024 citations recorded by the source corpus.

Distant supervised relation extraction has been widely used to ﬁnd novel relational facts from text. However, distant supervision inevitably accompanies with the wrong labelling problem, and these noisy data will substantially hurt the performance of relation extraction. To alleviate this issue, we propose a sentence-level attention-based model for relation extraction. In this model, we employ convolutional neural networks to embed the semantics of sentences. Afterwards, we build sentence-level attention over multiple instances, which is expected to dynamically reduce the weights of those noisy instances. Experimental results on real-world datasets show that, our model can make full use of all informative sentences and effectively reduce the inﬂuence of wrong labelled instances. Our model achieves signiﬁcant and consistent improvements on relation extraction as compared with baselines. The source code of this paper can be obtained from https: //github.com/thunlp/NRE .

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Denoising Distantly Supervised Open-Domain Question Answering (2018, background): “Lin et al. (2016) utilize selective attention to aggregate the information of all sentences to extract relational facts.”
ERNIE: Enhanced Language Representation with Informative Entities (2019, neutral): “are introduced into CNN (Zeng et al., 2015; Lin et al., 2016; Wu et al., 2017; Han et al., 2018b).”
Curriculum-Meta Learning for Order-Robust Continual Relation Extraction (2021, methodology): “The conventional relation extraction methods could be categorized into three domains by the way data is used: supervised methods (Zelenko, Aone, and Richardella 2002; Liu et al. 2013; Zeng et al. 2014; Lin et al. 2016; Miwa and Bansal 2016), semi-supervised methods (Chen et al.”
Continual Relation Learning via Episodic Memory Activation and Reconsolidation (2020, background): “The conventional RE methods (Riedel et al., 2013; Zeng et al., 2014; Lin et al., 2016) mostly focus on recognizing relations for a ﬁxed pre-deﬁned relation set, and cannot handle rapidly emerging novel relations in the real world.”
FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation (2018, methodology): “…is a primary approach to alleviate this problem for RC (Mintz et al.; Riedel et al.; Hoffmann et al., 2011; Surdeanu et al., 2012; Zeng et al., 2015; Lin et al., 2016), which heuristically aligns knowledge bases (KBs) and text to automatically annotate adequate amounts of training instances.”
Neural Snowball for Few-Shot Relation Learning (2019, background): “To further reduce data noise and highlight crucial instances, at-least-one (Zeng et al., 2015) and soft attention (Lin et al., 2016) have also been proposed.”
Assessing The Factual Accuracy of Generated Text (2019, methodology): “aches for fact extraction. One is a two-step process that first involves recognizing all the named entities in a sentence, and then classifying the relation for every pair of entities in the sentence [14, 32]. Our other approach is to use an end-to-end model with a Transformer-based architecture [36”
Towards Understanding Gender Bias in Relation Extraction (2019, methodology): “OpenNRE models combine methods including usage of selective attention to add weight to sentences with relevant information (Lin et al., 2016) as well as methods to reduce noise at an entity-pair level (Lin et al.”
DARE: Data Augmented Relation Extraction with GPT-2 (2020, background): “In this work, we focus on supervised RE (Zeng et al., 2014; Lin et al., 2016; Wu et al., 2017; Verga et al., 2018), where relation types come from a set of predeﬁned categories, as opposed to Open Information Extraction approaches that represent relations among entities using their surface forms…”
Distant Supervision Relation Extraction with Intra-Bag and Inter-Bag Attentions (2019, methodology): “4, where Mintz (Mintz et al., 2009), MultiR (Hoffmann et al., 2011) and MIMLRE (Surdeanu et al., 2012) are conventional feature-based methods, and (Lin et al., 2016) and (Liu et al., 2017) are PCNN-based ones 4 .”
Adversarial Training for Relation Extraction (2017, methodology): “An instance-level selective attention mechanism was introduced for MIML by Lin et al. (2016), and has signiﬁcantly improved the prediction accuracy for several of these base deep models.”
Generalizing to Unseen Entities and Entity Pairs with Row-less Universal Schema (2016, methodology): “Mintz et al. (2009) train per relation linear classifiers using features derived from the sentences in which the entity pair is mentioned.”
Context-Aware Representations for Knowledge Base Relation Extraction (2017, methodology): “Most of the methods (Riedel et al., 2010; Zeng et al., 2015; Lin et al., 2016) focus on predicting a single relation type based on the combined evidence from all of the occurrences of an entity pair.”
Prototypical Representation Learning for Relation Extraction (2021, neutral): “…-10.900574 32.073906 -7.5629783 36.904022 -15.117426 -47.537777 , various denoising methods have been explored to reduce noise effects for distant supervision (Zeng et al., 2015; Lin et al., 2016; Jiang et al., 2016; Han et al., 2018a; Wu et al., 2017; Qin et al., 2018a;b; Feng et al., 2018).”
End-to-End Human Pose and Mesh Reconstruction with Transformers (2020, background): “Attentions and Transformers: Recent studies [31, 23, 48] have shown that attention mechanisms improve the performance on various language tasks.”
Entity-Relation Extraction as Multi-Turn Question Answering (2019, methodology): “Many earlier entity-relation extraction systems are pipelined (Zelenko et al., 2003; Miwa et al., 2009; Chan and Roth, 2011; Lin et al., 2016): an entity extraction model ﬁrst identiﬁes entities of interest and a relation extraction model then constructs relations between the extracted entities.”
Hybrid Attention-Based Prototypical Networks for Noisy Few-Shot Relation Classification (2019, background): “Then, Lin et al. (2016) enhance it by adopting selective attention over instances, which benefits from all instances and meanwhile restrains the influence of wronglabeled ones.”