Incremental Prompting: Episodic Memory Prompt for Lifelong Event Detection

By Minqian Liu, Shiyu Chang, Lifu Huang. 2022

34 citations recorded by the source corpus.

Lifelong event detection aims to incrementally update a model with new event types and data while retaining the capability on previously learned old types. One critical challenge is that the model would catastrophically forget old types when continually trained on new data. In this paper, we introduce Episodic Memory Prompts (EMP) to explicitly retain the learned task-specific knowledge. Our method adopts continuous prompt for each task and they are optimized to instruct the model prediction and learn event-specific representation. The EMPs learned in previous tasks are carried along with the model in subsequent tasks, and can serve as a memory module that keeps the old knowledge and transferring to new tasks. Experiment results demonstrate the effectiveness of our method. Furthermore, we also conduct a comprehensive analysis of the new and old event types in lifelong learning.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Health Information Management, Natural Language Processing, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Hang Su
2. Jun Zhu
3. Liyuan Wang
4. Xingxing Zhang
5. Zhiwu Huang
6. Yabin Wang
7. Xiaopeng Hong
8. Tianyu Pang

In Computer Vision and Pattern Recognition

1. Hang Su
2. Jun Zhu
3. Liyuan Wang
4. Xingxing Zhang
5. Zhiwu Huang
6. Yabin Wang
7. Xiaopeng Hong
8. Han-Jia Ye

In Health Information Management

1. Zhiwu Huang
2. Yabin Wang
3. Xiaopeng Hong

In Natural Language Processing

1. Liyuan Wang
2. Jun Zhu
3. Zhiwu Huang
4. Hang Su
5. Yabin Wang
6. Xingxing Zhang
7. Xiaopeng Hong

In Meta & Continual Learning

1. Xingxing Zhang
2. Liyuan Wang
3. Hang Su
4. Jun Zhu
5. Zhiwu Huang
6. Yabin Wang
7. Xiaopeng Hong
8. Han-Jia Ye

Citations in the corpus

• A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “For example, a pre-trained transformer can effectively accommodate downstream tasks by learning only a few task-specific parameters, including adaptor-tuning [165] with inserted fully-connected layers (CTR [208], B-CL [209], CLASSIC [207], CPT [205], CLIF [190], AdapterCL [282], ACM [499], ADA [111]”
• S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learning (2022, methodology): “The prompting method [42] suggests learning episodic memory prompts to explicitly carry previously learned knowledge to subsequent tasks for class-incremental event detection.”