KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction

By Xiang Chen, Ningyu Zhang, Ningyu Zhang, Xin Xie, Shumin Deng, Yunzhi Yao, Chuanqi Tan, Fei Huang, Luo Si, Huajun Chen. 2021

507 citations recorded by the source corpus.

Recently, prompt-tuning has achieved promising results for specific few-shot classification tasks. The core idea of prompt-tuning is to insert text pieces (i.e., templates) into the input and transform a classification task into a masked language modeling problem. However, for relation extraction, determining an appropriate prompt template requires domain expertise, and it is cumbersome and time-consuming to obtain a suitable label word. Furthermore, there exists abundant semantic and prior knowledge among the relation labels that cannot be ignored. To this end, we focus on incorporating knowledge among relation labels into prompt-tuning for relation extraction and propose a Knowledge-aware Prompt-tuning approach with synergistic optimization (KnowPrompt). Specifically, we inject latent knowledge contained in relation labels into prompt construction with learnable virtual type words and answer words. Then, we synergistically optimize their representation with structured constraints. Extensive experimental results on five datasets with standard and low-resource settings demonstrate the effectiveness of our approach. Our code and datasets are available in GitHub1 for reproducibility.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Graham Neubig
2. Pengfei Liu
3. Weizhe Yuan
4. Jinlan Fu
5. Mohit Bansal
6. Zhengbao Jiang
7. Lifu Huang
8. Dan Roth

In Information Systems

1. Pengfei Liu
2. Graham Neubig
3. Lifu Huang
4. Weizhe Yuan
5. Zhengbao Jiang
6. Dan Roth
7. Fei Huang
8. Eneko Agirre

In Natural Language Processing

1. Graham Neubig
2. Pengfei Liu
3. Jinlan Fu
4. Weizhe Yuan
5. Dan Roth
6. Zhengbao Jiang
7. Eneko Agirre
8. Fei Huang

Citations in the corpus

• Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (2021, methodology): “Existing works [13] make a step by making prompts with additional marks to encode lexical information, such as entity markings.”
• Incremental Prompting: Episodic Memory Prompt for Lifelong Event Detection (2022, background): “Prompt Learning Conditioning on large-scale pre-trained language models, prompt learning (Brown et al., 2020; Lester et al., 2021; Chen et al., 2021; Liu et al., 2021; Wang et al., 2021a)”
• Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners (2021, background): “Thus, recent studies [34, 15, 7] conducted in this field have been focused on automatically searching the prompts.”
• Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey (2021, background): “[17], instead of using rules, constructed the prompts by leveraging learnable virtual template words and virtual answer words.”
• Retrieval And Structuring Augmented Generation with Large Language Models (2025, unknown): “Prompt-tuning models [29, 71] also improve relational reasoning by integrating diverse prompts.”