Hierarchical Relation Extraction with Coarse-to-Fine Grained Attention

By Xu Han, Pengfei Yu, Zhiyuan Liu, Maosong Sun, Peng Li. 2018

146 citations recorded by the source corpus.

Distantly supervised relation extraction employs existing knowledge graphs to automatically collect training data. While distant supervision is effective to scale relation extraction up to large-scale corpora, it inevitably suffers from the wrong labeling problem. Many efforts have been devoted to identifying valid instances from noisy data. However, most existing methods handle each relation in isolation, regardless of rich semantic correlations located in relation hierarchies. In this paper, we aim to incorporate the hierarchical information of relations for distantly supervised relation extraction and propose a novel hierarchical attention scheme. The multiple layers of our hierarchical attention scheme provide coarse-to-fine granularity to better identify valid instances, which is especially effective for extracting those long-tail relations. The experimental results on a large-scale benchmark dataset demonstrate that our models are capable of modeling the hierarchical information of relations and significantly outperform other baselines. The source code of this paper can be obtained from https://github.com/thunlp/HNRE.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Zhiyuan Liu
2. Maosong Sun
3. Philip S. Yu
4. Xu Han
5. Erik Cambria
6. Jie Zhou
7. Yankai Lin
8. Peng Li

Citations in the corpus

• Imbalanced Continual Learning with Partitioning Reservoir Sampling (2020, background): “Other works such as [34] and [95] define the minority classes as less than 100 or 200 samples in the training set, respectively.”
• Continual Relation Learning via Episodic Memory Activation and Reconsolidation (2020, methodology): “…et al., 2015; Miwa and Bansal, 2016) and distantly supervised models (Bunescu and Mooney, 2007; Mintz et al., 2009; Riedel et al., 2010; Hoffmann et al., 2011; Zeng et al., 2015; Lin et al., 2016; Han et al., 2018a; Baldini Soares et al., 2019), focuses on extracting predefined relations from text.”
• More Data, More Relations, More Context and More Openness: A Review and Outlook for Relation Extraction (2020, methodology): “…to denoise DS data has also been explored, such as incorporating KGs as external information to guide instance selection (Ji et al., 2017; Han et al., 2018b; Zhang et al., 2019a; Qu et al., 2019) and adopting multi-lingual corpora for the information consistency and complementarity…”
• Long-tail Relation Extraction via Knowledge Graph Embeddings and Graph Convolution Networks (2019, methodology): “Inspired by the rich knowledge in KGs, recent works (Han et al., 2018a; Wang et al., 2018; Lei et al., 2018) extend DS models under the guidance of KGs.”
• ERNIE: Enhanced Language Representation with Informative Entities (2019, methodology): “Since FewRel is built by checking whether the sentences contain facts in Wiki-data, we drop the related facts in KGs before pre-training for fair comparison.”
• A Survey on Knowledge Graphs: Representation, Acquisition, and Applications (2020, methodology): “APCNN [133] introduces entity description by PCNN and sentence-level attention, while HATT [134] proposes hierarchical selective attention to capture the relation hierarchy by concatenating attentive representation of each hierarchical layer.”
• A Comprehensive Survey on Automatic Knowledge Graph Construction (2023, methodology): “PCNN+pooling positionembeddings TM[190] PCNN+Transitionmatrix+Bagembedding positionembeddings HAtt[189] CNN/PCNN+hierarchicalattention positionembeddings,relationhierarchy Enhancedlearningstrategy-based RL-based[192] RL-baseddataredistributor+CNN/PCNN+Result-drivenreward positionembeddings DSGAN[193”
• Improving Relation Extraction by Pre-trained Language Representations (2019, methodology): “Relation extraction models benefit from efficient representations of long-term dependencies [Zhang et al., 2018] and hierarchical relation types [Han et al., 2018].”
• DocRED: A Large-Scale Document-Level Relation Extraction Dataset (2019, background): “In recent years, various neural models have been explored to encode relational patterns of entities for sentence-level RE, and achieve state-of-theart performance (Socher et al., 2012; Zeng et al., 2014, 2015; dos Santos et al., 2015; Xiao and Liu, 2016; Cai et al., 2016; Lin et al., 2016; Wu et al.”