Prototypical Representation Learning for Relation Extraction

By Ning Ding, Xiaobin Wang, Yao Fu, Guangwei Xu, Pengjun Xie, Ying Shen, Fei Huang, Haitao Zheng, Rui Zhang. 2021

67 citations recorded by the source corpus.

Recognizing relations between entities is a pivotal task of relational learning. Learning relation representations from distantly-labeled datasets is difficult because of the abundant label noise and complicated expressions in human language. This paper aims to learn predictive, interpretable, and robust relation representations from distantly-labeled data that are effective in different settings, including supervised, distantly supervised, and few-shot learning. Instead of solely relying on the supervision from noisy labels, we propose to learn prototypes for each relation from contextual information to best explore the intrinsic semantics of relations. Prototypes are representations in the feature space abstracting the essential semantics of relations between entities in sentences. We learn prototypes based on objectives with clear geometric interpretation, where the prototypes are unit vectors uniformly dispersed in a unit ball, and statement embeddings are centered at the end of their corresponding prototype vectors on the surface of the ball. This approach allows us to learn meaningful, interpretable prototypes for the final classification. Results on several relation learning tasks show that our model significantly outperforms the previous state-of-the-art models. We further demonstrate the robustness of the encoder and the interpretability of prototypes with extensive experiments.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Zhiyuan Liu
2. Maosong Sun
3. Minlie Huang
4. Xu Han
5. Xipeng Qiu
6. Jie Tang
7. Ji-rong Wen
8. Wayne Xin Zhao

Citations in the corpus

• Pre-Trained Models: Past, Present and Future (2021, methodology): “Moreover, efficient and effective task-specific fine-tuning is also an important research direction for the future application of PTMs (Soares et al., 2019; Ding et al., 2021b).”
• PTR: Prompt Tuning with Rules for Text Classification (2021, background): “, 2020), to text classification (Baldini Soares et al., 2019; Peng et al., 2020; Ding et al., 2021), fine-tuned PLMs have been demonstrated their effectiveness on almost all important NLP tasks.”
• Refining Sample Embeddings with Relation Prototypes to Enhance Continual Relation Extraction (2021, background): “In addition, in order to accurately represent relations with limited samples, the idea of prototypical networks is introduced into RE(Gao et al., 2019; Ding et al., 2021).”
• Few-NERD: A Few-shot Named Entity Recognition Dataset (2021, background): “ProtoBERT Inspired by achievements of metalearning approaches (Finn et al., 2017; Snell et al., 2017; Ding et al., 2021) on few-shot learning.”
• CLINE: Contrastive Learning with Semantic Negative Examples for Natural Language Understanding (2021, unknown): “…(Velickovic et al., 2019), visual representations (van den Oord et al., 2018; He et al., 2020; Chen et al., 2020), response representations (Lin et al., 2020b; Su et al., 2020), text representations (Iter et al., 2020; Ding et al., 2021) and structured world models (Kipf et al., 2020).”