DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning
By Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Chen-Yu Lee, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer G. Dy, Tomas Pfister. 2022
880 citations recorded by the source corpus.
Continual learning aims to enable a single model to learn a sequence of tasks without catastrophic forgetting. Top-performing methods usually require a rehearsal buffer to store past pristine examples for experience replay, which, however, limits their practical value due to privacy and memory constraints. In this work, we present a simple yet effective framework, DualPrompt, which learns a tiny set of parameters, called prompts, to properly instruct a pre-trained model to learn tasks arriving sequentially without buffering past examples. DualPrompt presents a novel approach to attach complementary prompts to the pre-trained backbone, and then formulates the objective as learning task-invariant and task-specific"instructions". With extensive experimental validation, DualPrompt consistently sets state-of-the-art performance under the challenging class-incremental setting. In particular, DualPrompt outperforms recent advanced continual learning methods with relatively large buffer sizes. We also introduce a more challenging benchmark, Split ImageNet-R, to help generalize rehearsal-free continual learning research. Source code is available at https://github.com/google-research/l2p.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Prompting & Reasoning, Meta & Continual Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Bernt Schiele
- Han-Jia Ye
- Trevor Darrell
- De-chuan Zhan
- Alan L. Yuille
- Ziwei Liu
In Prompting & Reasoning
In Meta & Continual Learning
Citations in the corpus
- A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, methodology): “…optimizing a weighted summation of the prompt pool with attention factors (CODA-Prompt [206]), using explicitly task-sharing and task-specific prompts (DualPrompt [207]) or only task-specific prompts (S-Prompts [208]), progressive expansion of task-specific prompts (Progressive Prompts [209]), etc.”
- First Session Adaptation: A Strong Replay-Free Baseline for Class-Incremental Learning (2023, methodology): “Current SOTA methods for class incremental learning start from a pre-trained backbone [13, 19, 35, 34] and then adapt the features at each session of continual learning.”
- SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Model (2023, methodology): “DualPrompt [45] extended this idea by attaching complementary prompts to the pre-trained representation layer for learning task-invariant and task-specific instructions.”
- CODA-Prompt: COntinual Decomposed Attention-Based Prompting for Rehearsal-Free Continual Learning (2022, methodology): “In this paper, we instead consider the highly-impactful and well-established setting of rehearsal-free continual learning [38, 56, 57, 65, 66]1, limiting our scope to strategies for continual learning which do not store training data.”
- A Simple Baseline that Questions the Use of Pretrained-Models in Continual Learning (2022, background): “Split ImageNet-R is a newly proposed dataset for continual learning by [26].”
- S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learning (2022, background): “There have been also emerging some other transformer-based incremental learning methods like [71, 27, 37], and other incremental prompting methods like [42, 65].”
- One Size Fits All for Semantic Shifts: Adaptive Prompt Tuning for Continual Learning (2023, methodology): “Meanwhile, DP (Wang et al., 2022b) employs the simultaneous use of both universal and specific prompts for all tasks.”
- Continual Segmentation with Disentangled Objectness Learning and Class Recognition (2024, neutral): “Recently, more works [13, 37, 41, 41, 49, 50] started to leverage the dynamic nature of Transformers.”
- Deep Class-Incremental Learning: A Survey (2023, background): “CIL with Pre-trained Models: Recently, pre-trained models have shown to work competitively with their strong transferability, especially for ViT-based methods [72], [73], [74], [75].”
- Convolutional Prompting meets Language Models for Continual Learning (2024, methodology): “While existing works [53, 54] have shown to perform well by prompting a pre-trained transformer, they rely on a single set of prompt vectors needing to compress all necessary information of an image into one single set.”
- Class-Incremental Learning: A Survey (2023, methodology): “Similarly, L2P [69] and DualPrompt [70] explore how to build CIL learners with pre-trained ViT.”
- ECLIPSE: Efficient Continual Learning in Panoptic Segmentation with Visual Prompt Tuning (2024, background): “Namely, L2P [45] and DualPrompt [44] freeze the pre-trained model and select the most relevant prompts from a prompts pool in a key-value mechanism.”
- InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learning (2024, background): “03 DualPrompt [6] 35 .”
- Consistent Prompting for Rehearsal-Free Continual Learning (2024, neutral): “Competitors We focus on comparing our model with the state-of-the-art prompt-based methods, i.e., L2P [46], Du-alPrompt [45], ESN [44] and CODAprompt [36].”
- A Closer Look at Rehearsal-Free Continual Learning * (2022, methodology): “[54, 60, 61].”