Distant Supervision for Relation Extraction via Piecewise Convolutional Neural Networks

By Daojian Zeng, Kang Liu, Yubo Chen, Jun Zhao. 2015

1,111 citations recorded by the source corpus.

Two problems arise when using distant supervision for relation extraction. First, in this method, an already existing knowledge base is heuristically aligned to texts, and the alignment results are treated as labeled data. However, the heuristic alignment can fail, resulting in wrong label problem. In addition, in previous approaches, statistical models have typically been applied to ad hoc features. The noise that originates from the feature extraction process can cause poor performance. In this paper, we propose a novel model dubbed the Piecewise Convolutional Neural Networks (PCNNs) with multi-instance learning to address these two problems. To solve the first problem, distant supervised relation extraction is treated as a multi-instance problem in which the uncertainty of instance labels is taken into account. To address the latter problem, we avoid feature engineering and instead adopt convolutional architecture with piecewise max pooling to automatically learn relevant features. Experiments show that our method is effective and outperforms several competitive baseline methods.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Continual Few-shot Relation Learning via Embedding Space Regularization and Data Augmentation (2022, background): “…ones and analyse our results thoroughly.117 2 Related Work118 Conventional RE methods include supervised (Ze-119 lenko et al., 2002; Liu et al., 2013; Zeng et al., 2014;120 Miwa and Bansal, 2016), semi-supervised (Chen121 et al., 2006; Sun et al., 2011; Hu et al., 2020) and122 distantly…”
Continual Relation Learning via Episodic Memory Activation and Reconsolidation (2020, background): “…al., 2015; Miwa and Bansal, 2016) and distantly supervised models (Bunescu and Mooney, 2007; Mintz et al., 2009; Riedel et al., 2010; Hoffmann et al., 2011; Zeng et al., 2015; Lin et al., 2016; Han et al., 2018a; Bal-dini Soares et al., 2019), focuses on extracting pre-deﬁned relations from text.”
FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation (2018, methodology): “Under this setting, we evaluate different few-shot training strategies and state-of-the-art few-shot learning methods built upon two widely used instance encoders, CNN (Zeng et al., 2014) and PCNN (Zeng et al., 2015).”
Neural Snowball for Few-Shot Relation Learning (2019, background): “To further reduce data noise and highlight crucial instances, at-least-one (Zeng et al., 2015) and soft attention (Lin et al., 2016) have also been proposed.”
Towards Understanding Gender Bias in Relation Extraction (2019, methodology): “To convert sentences into vectors, researchers propose convolutional neural networks as well as the pieceweise convoultional neural networks (PCNN) which retain more structural information between entities (Zeng et al., 2015).”
Neural Relation Extraction with Selective Attention over Instances (2016, methodology): “(Zeng et al., 2015) follows the assumption that at least one mention of the entity pair will reflect their relation, and only uses the sentence with the highest probability in each set for training.”
A Multi-Channel Deep Neural Network for Relation Extraction (2020, background): “[30] proposed a piecewise max pooling layer, which is implemented on different parts of a sentence divided by two named entities.”
Context-Aware Representations for Knowledge Base Relation Extraction (2017, background): “Most of the methods (Riedel et al., 2010; Zeng et al., 2015; Lin et al., 2016) focus on predicting a single relation type based on the combined evidence from all of the occurrences of an entity pair.”
Reinforcement Learning for Relation Classification From Noisy Data (2018, methodology): “To address the issue of noisy labeling, previous studies adopt multi-instance learning to consider the noises of instances (Riedel, Yao, and McCallum 2010; Hoffmann et al. 2011; Surdeanu et al. 2012; Zeng et al. 2015; Lin et al. 2016; Ji et al. 2017).”
Learning to Annotate: Modularizing Data Augmentation for Text Classifiers with Natural Language Explanations (2019, methodology): “(2) PCNN (Zeng et al., 2015) uses piece-wise max-pooling to aggregate CNNgenerated features.”
NERO: A Neural Rule Grounding Framework for Label-Efficient Relation Extraction (2020, methodology): “We apply the following methods on Smatched: (1) PCNN [36] represents each token using both word embeddings and positional embeddings.”
Quantifying Similarity between Relations with Fact Distribution (2019, background): “, 2017) and relation extraction (Riedel et al., 2013; Liu et al., 2013; Zeng et al., 2014; Santos et al., 2015; Zeng et al., 2015; Lin et al., 2016), and relation prediction (Bordes et al.”
Fidelity-Weighted Learning (2017, methodology): “…or noisy labels can be leveraged by employing a particular architecture or deﬁning a proper loss function to avoid over-ﬁtting to imperfections of the training data (Dehghani et al., 2017b; Vahdat, 2017; Patrini et al., 2017; Beigman & Klebanov, 2009; Zeng et al., 2015; Bunescu & Mooney, 2007).”
Improving Relation Extraction by Pre-trained Language Representations (2019, methodology): “We evaluate the PCNN implementation of Zeng et al. [2015] 4 on the two datasets and use it as a state-of-the-art baseline in our analysis section.”
ERNIE: Enhanced Language Representation with Informative Entities (2019, background): “These extra mark tokens play a similar role like position embeddings in the conventional relation classiﬁcation models (Zeng et al., 2015).”
A Comprehensive Survey on Automatic Knowledge Graph Construction (2023, background): “[197] CNN/PCNN+Intra/innerbagattention pre-trainedmodel Yuanetal.”
Learning Dual Retrieval Module for Semi-supervised Relation Extraction (2019, methodology): “ds that can automatically extract entity relationships from text with less human supervision. In recent years there has been a surge of interest in applying distant supervision to relation extraction [19, 22, 37, 38]. The general idea is to leverage facts stored in external knowledge bases (KBs) to ”
Relation Extraction Using Distant Supervision (2018, methodology): “The authors of [73] show that the same neural architecture but with single max-pooling performs similar to MultiR and MIML − RE.”
Refining Sample Embeddings with Relation Prototypes to Enhance Continual Relation Extraction (2021, background): “…and utilizing various deep neural networks to discover the relations be-tween entities given contexts, including: (1) Convolutional neural networks (CNNs) (Zeng et al., 2014, 2015; Nguyen and Grishman, 2015; Lin et al., 2016; Ji et al., 2017) can effectively extract local textual features.”