Learning to Prompt for Continual Learning
By Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer G. Dy, Tomas Pfister. 2021
1,330 citations recorded by the source corpus.
The mainstream paradigm behind continual learning has been to adapt the model parameters to non-stationary data distributions, where catastrophic forgetting is the central challenge. Typical methods rely on a rehearsal buffer or known task identity at test time to retrieve learned knowl-edge and address forgetting, while this work presents a new paradigm for continual learning that aims to train a more succinct memory system without accessing task identity at test time. Our method learns to dynamically prompt (L2P) a pre-trained model to learn tasks sequen-tially under different task transitions. In our proposed framework, prompts are small learnable parameters, which are maintained in a memory space. The objective is to optimize prompts to instruct the model prediction and ex-plicitly manage task-invariant and task-specific knowledge while maintaining model plasticity. We conduct comprehen-sive experiments under popular image classification bench-marks with different challenging continual learning set-tings, where L2P consistently outperforms prior state-of-the-art methods. Surprisingly, L2P achieves competitive results against rehearsal-based methods even without a re-hearsal buffer and is directly applicable to challenging task-agnostic continual learning. Source code is available at https://github.com/google-research/12p.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Prompting & Reasoning, Meta & Continual Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Bernt Schiele
- Trevor Darrell
- Yi Yang
- Xiaogang Wang
- Andrew Zisserman
In Prompting & Reasoning
In Meta & Continual Learning
Citations in the corpus
- Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning (2022, methodology): “Prompt tuning has been applied to various applications for NLP including continual learning [59], model robustness [60, 61], summarization [62], machine translation [63], co-training [64], probing language models [65, 65], inverse prompting [66] and transfer learning [67].”
- Visual Prompt Tuning (2022, methodology): “Due to the disparity between vision and language modalities, in this paper we ask: can the same method can be applied successfully to image encoders? We are the first work (see related concurrent works [52,62,11,2]) to tackle this question and investigate the generality and feasibility of visual pro”
- A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, methodology): “Representative strategies include selecting the most relevant prompts from a prompt pool (L2P [205]), optimizing a weighted summation of the prompt pool with attention factors (CODA-Prompt [206]), using explicitly task-sharing and task-specific prompts (DualPrompt [207]) or only task-specific prompts…”
- Learnability and Algorithm for Continual Learning (2023, background): “As pre-training models or feature extractors are increasingly used in all types of applications, including continual learning (Wang et al., 2022c; Kim et al., 2022a; Wu et al., 2022), we also take this approach.”
- First Session Adaptation: A Strong Replay-Free Baseline for Class-Incremental Learning (2023, methodology): “Current SOTA methods for class incremental learning start from a pre-trained backbone [13, 19, 35, 34] and then adapt the features at each session of continual learning.”
- SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Model (2023, methodology): “Inspired by the techniques of using pre-trained knowledge in NLP, L2P [46] employed an additional set of learnable parameters called “prompts” that dynamically instruct a pre-trained representation layer to learn incremental tasks.”
- CODA-Prompt: COntinual Decomposed Attention-Based Prompting for Rehearsal-Free Continual Learning (2022, methodology): “In this paper, we instead consider the highly-impactful and well-established setting of rehearsal-free continual learning [39, 57, 58, 66, 67] 1 , limiting our scope to strategies for continual learning which do not store training data.”
- A Simple Baseline that Questions the Use of Pretrained-Models in Continual Learning (2022, methodology): “Concretely, our baseline achieves 88.53% with a 0 buffer size, which is more than 4.7% and 2.22% better than the runner-up and significantly more complicated L2P with no exemplars and L2P with 50 exemplars/class buffer, respectively.”
- S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learning (2022, methodology): “For CORe50 and DomainNet, we follow [66] and [18] to present the average forward classification accuracy.”
- Incremental Prompting: Episodic Memory Prompt for Lifelong Event Detection (2022, background): “Lifelong Event Detection Deep neural networks have shown state-of-the-art performance on supervised event detection (Nguyen et al., 2016; Feng et al., 2016; Zhang et al., 2017; Huang and Ji, 2020; Wang et al., 2021b).”
- DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning (2022, methodology): “To ensure fair comparison, every aforementioned methods start from the same ImageNet pre-trained ViT-B/16 [10], following the setting in [60].”
- RSPrompter: Learning to Prompt for Remote Sensing Instance Segmentation Based on Visual Foundation Model (2023, methodology): “Current methods for constructing prompts primarily involve manual templates, heuristic-based templates, generation, word embedding fine-tuning, and pseudotokens [69], [80].”
- Orthogonal Subspace Learning for Language Model Continual Learning (2023, methodology): “To compare O-LoRA to the recent CL approaches (Wang et al., 2022c; Qin and Joty, 2021), we use the pre-trained T5-large model.”
- Continual Learning on a Diet: Learning from Sparsely Labeled Streams Under Constrained Computation (2024, methodology): “Then, we report the accuracy of the union test sets in the last time step T as A ( T ) = a T following Chaudhry et al. (2019a); Fini et al. (2022); Wang et al. (2022).”
- Consistent Prompting for Rehearsal-Free Continual Learning (2024, positive): “Competitors We focus on comparing our model with the state-of-the-art prompt-based methods, i.e., L2P [46], Du-alPrompt [45], ESN [44] and CODAprompt [36].”