Continual Learning for Named Entity Recognition

By Natawut Monaikul, Giuseppe Castellucci, Simone Filice, Oleg Rokhlenko. 2021

91 citations recorded by the source corpus.

Named Entity Recognition (NER) is a vital task in various NLP applications. However, in many real-world scenarios (e.g., voice-enabled assistants) new named entities are frequently introduced, entailing re-training NER models to support these new entities. Re-annotating the original training data for the new entities could be costly or even impossible when storage limitations or security concerns restrict access to that data, and annotating a new dataset for all of the entities becomes impractical and error-prone as the number of entities increases. To tackle this problem, we introduce a novel Continual Learning approach for NER, which requires new training material to be annotated only for the new entities. To preserve the existing knowledge previously learned by the model, we exploit the Knowledge Distillation (KD) framework, where the existing NER model acts as the teacher for a new NER model (i.e., the student), which learns the new entity by using the new training material and retains knowledge of old entities by imitating the teacher's outputs on this new training set. Our experiments show that this approach allows the student model to ``progressively'' learn to identify new entities without forgetting the previously learned ones. We also present a comparison with multiple strong baselines to demonstrate that our approach is superior for continually updating an NER model.

Subfields: Artificial Intelligence, Information Extraction, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Hang Su
2. Jun Zhu
3. Liyuan Wang
4. Xingxing Zhang
5. Gautam M. Shroff
6. Ricardo Henao
7. Tong Yu
8. Vaibhav Varshney

In Information Extraction

1. Jun Zhu

In Meta & Continual Learning

1. Xingxing Zhang
2. Liyuan Wang
3. Hang Su
4. Jun Zhu
5. Gautam M. Shroff
6. Vaibhav Varshney
7. Lovekesh Vig
8. Mayur Patidar

Citations in the corpus

• A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “The applications of continual learning in NLP are also highly diverse, such as dialogue system [130], [131], [260], [260], [282], [298], [437], text classification [87], [172], [325], [460], sentence generation [246], [298], [499], relation learning [343], [357], [455], [465], neural machine transla”
• Prompt Augmented Generative Replay via Supervised Contrastive Learning for Lifelong Intent Detection (2022, background): “Shin et al. (2017) provide a Generative Replay based approach where pseudo-labelled data corresponding to the prior task is generated and interleaved with data from the current task to train a solver whereas van de Ven et al. (2020) use the same model as both a solver and generator and instead of re”
• Few-Shot Class-Incremental Learning for Named Entity Recognition (2022, methodology): “Especially, CI NER (Monaikul et al., 2021) has the worst result among all the methods.”