Template-Based Named Entity Recognition Using BART

By Leyang Cui, Yu Wu, Jian Liu, Sen Yang, Yue Zhang. 2021

403 citations recorded by the source corpus.

There is a recent interest in investigating few-shot NER, where the low-resource target domain has different label sets compared with a resource-rich source domain. Existing methods use a similarity-based metric. However, they cannot make full use of knowledge transfer in NER model parameters. To address the issue, we propose a template-based method for NER, treating NER as a language model ranking problem in a sequence-to-sequence framework, where original sentences and statement templates filled by candidate named entity span are regarded as the source sequence and the target sequence, respectively. For inference, the model is required to classify each candidate span based on the corresponding template scores. Our experiments demonstrate that the proposed method achieves 92.55% F1 score on the CoNLL03 (rich-resource task), and significantly better than fine-tuning BERT 10.88%, 15.34%, and 11.73% F1 score on the MIT Movie, the MIT Restaurant, and the ATIS (low-resource task), respectively.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (2021, methodology): “As noted above, there are two main varieties of prompts: cloze prompts [17, 100], which ill in the blanks of a textual string (e.”
Few-Shot Class-Incremental Learning for Named Entity Recognition (2022, methodology): “Recently, (Cui et al., 2021) proposes Template NER, a few-shot friendly model for NER that convert NER into a sequence-to-sequence problem.”
Reframing Instructional Prompts to GPTk’s Language (2021, background): “, 2021) and named entity recognition (Cui et al., 2021) tasks, respectively.”
AdaPrompt: Adaptive Model Training for Prompt-based NLP (2022, methodology): “Second, to project label words to task labels, most existing work (Schick and Schütze, 2021a,b; Cui et al., 2021) uses a pre-deﬁned verbalizer.”
Large language models are few-shot clinical information extractors (2022, background): “For example, given a candidate entity X and full model access, the entity type can be predicted via an argmax over the possible types Y of the probability of statements like “X is a Y entity” (Cui et al., 2021).”
Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey (2021, background): “[26] explore generation-based ranking for NER, especially in few-shot and cross-domain few-shot settings.”
InferFix: End-to-End Program Repair with LLMs (2023, methodology): “[9] investigate a template-based method for exploiting the few-shot learning potential of generative pre-trained language models to sequence labeling.”
Do Prompts Solve NLP Tasks Using Natural Language? (2022, methodology): “For sentence classiﬁcation, we follow Gao et al. (2021) to adopt RoBERTa-large (Liu et al., 2019) and conduct experiments on eight sentence classiﬁcation datasets For structure prediction, we use the method and setting of Cui et al. (2021), which formulized NER as a text generation task.”
How Many Data Samples is an Additional Instruction Worth? (2022, background): “, 2020), Information Extraction (IE) (Chen et al., 2021; Cui et al., 2021) and many more (Liu et al.”
A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities (2022, methodology): “Different tasks [107] could have different blanks positions to test whether the model has learned the corresponding semantic knowledge [107, 111].”
Unified Structure Generation for Universal Information Extraction (2022, methodology): “The generation can be enhanced using label templates (Li et al., 2021a; Liu et al., 2021; Cui et al., 2021), schema (Lu et al., 2021; Ahmad et al., 2021), and augmented language methods (Paolini et al., 2021).”
A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models (2024, neutral): “In this section, we will introduce important techniques involved in each process.”