Enriching Pre-trained Language Model with Entity Information for Relation Classification

By Shanchan Wu, Yifan He. International Conference on Information and Knowledge Management, 2019

463 citations recorded by the source corpus.

Relation classification is an important NLP task to extract relations between entities. The state-of-the-art methods for relation classification are primarily based on Convolutional or Recurrent Neural Networks. Recently, the pre-trained BERT model achieves very successful results in many NLP classification / sequence labeling tasks. Relation classification differs from those tasks in that it relies on information of both the sentence and the two target entities. In this paper, we propose a model that both leverages the pre-trained BERT language model and incorporates information from the target entities to tackle the relation classification task. We locate the target entities and transfer the information through the pre-trained architecture and incorporate the corresponding encoding of the two entities. We achieve significant improvement over the state-of-the-art method on the SemEval-2010 task 8 relational dataset.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/50714786ac191e796d8963b66364fbf46ac54761

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Zhiyuan Liu
2. Maosong Sun
3. Xu Han
4. Jie Zhou
5. Mohit Bansal
6. Yankai Lin
7. Tianyu Gao
8. Peng Li

Citations in the corpus

• Refining Sample Embeddings with Relation Prototypes to Enhance Continual Relation Extraction (2021, methodology): “Inspired by current CRE models (Wu and He, 2019; Han et al., 2020), we adopt an episodic memory module to store typical samples of relations that the model has learned in former tasks.”
• AdaPrompt: Adaptive Prompt-based Finetuning for Relation Extraction (2021, background): “Several works show that BERT-based models outperform both RNN and graph-based models with a large margin (Wu and He, 2019; Joshi et al., 2020; Yu et al., 2020).”
• More Data, More Relations, More Context and More Openness: A Review and Outlook for Relation Extraction (2020, methodology): “Compared with SRE models, NRE methods can effectively capture textual information and generalize to wider range of data.”
• RadGraph: Extracting Clinical Entities and Relations from Radiology Reports (2021, methodology): “Our Baseline approach to entity and relation extraction uses a BERT [31] model with a linear classification head on top of the last layer for NER and R-BERT [32] for relation extraction.”