Learn-Prune-Share for Lifelong Learning
By Zifeng Wang, Tong Jian, Kaushik R. Chowdhury, Yanzhi Wang, Jennifer G. Dy, Stratis Ioannidis. 2020
50 citations recorded by the source corpus.
In lifelong learning, we wish to maintain and update a model (e.g., a neural network classifier) in the presence of new classification tasks that arrive sequentially. In this paper, we propose a learn-prune-share (LPS) algorithm which addresses the challenges of catastrophic forgetting, parsimony, and knowledge reuse simultaneously. LPS splits the network into task-specific partitions via an ADMM-based pruning strategy. This leads to no forgetting, while maintaining parsimony. Moreover, LPS integrates a novel selective knowledge sharing scheme into this ADMM optimization framework. This enables adaptive knowledge sharing in an end-to-end fashion. Comprehensive experimental results on two lifelong learning benchmark datasets and a challenging real world radio frequency fingerprinting dataset are provided to demonstrate the effectiveness of our approach. Our experiments show that LPS consistently outperforms multiple state-of-the-art competitors.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Fajie Yuan
- Xiangnan He
- Zizhao Zhang
- Alexandros Karatzoglou
- Joemon M. Jose
- Tomas Pfister
- Sayna Ebrahimi
- Jennifer G. Dy
In Computer Vision and Pattern Recognition
- Zizhao Zhang
- Luc Van Gool
- Trevor Darrell
- Dacheng Tao
- Tomas Pfister
- Han Zhang
- Xiangnan He
- Sayna Ebrahimi
Citations in the corpus
- DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning (2022, methodology): “These methods can be further categorized as expanding the model [51, 65, 26, 31, 59], or dividing the model [35, 52, 58, 17, 11].”
- Learning to Prompt for Continual Learning (2021, background): “Nevertheless, these methods require substantial amount of additional parameters, sometimes close to the size of the full model [19,59].”
- One Person, One Model, One World: Learning Continual User Representation without Forgetting (2020, background): “We notice that similar attempts have been recently proven effective for solving problems in other research fields [7, 39].”