Long Live the Lottery: The Existence of Winning Tickets in Lifelong Learning

By Tianlong Chen, Zhenyu (Allen) Zhang, Sijia Liu, Shiyu Chang, Zhangyang Wang. 2021

72 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Recent Advances of Continual Learning in Computer Vision: An Overview (2021, neutral): “[213] adapted the lottery ticket hypothesis [214] to continual learning.”
Playing Lottery Tickets with Vision and Language (2021, background): “The properties of LTH are widely studied for image classification [44, 15, 18, 58, 68, 74, 47], and recently, the observations of LTH have also been evidenced across other fields, such as natural language processing [19, 76, 54, 6, 9], object detection [21], generative adversarial networks [10, 31, ”
A Survey on Deep Neural Network Pruning: Taxonomy, Comparison, Analysis, and Recommendations (2023, unknown): “Some works ([147], [221], [222], [223]) use KD to train the pruned networks.”
On the Soft-Subnetwork for Few-shot Class Incremental Learning (2022, background): “LL-Tickets [6] shows the existence of a sparse subnetwork, called lifelong tickets, that performs well on all tasks during continual learning.”
Forget-free Continual Learning with Winning Subnetworks (2022, background): “Very recently, LL-Tickets (Chen et al., 2021) shows the existence of a sparse subnetwork, called lifelong tickets, that performs well on all tasks during continual learning.”
How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition (2022, background): “Chen et al. (2021) extended this hypothesis to the lifelong setting by using a pruning and regrowing approach, in combination with functional regularization from unlabeled data from public sources.”
Queried Unlabeled Data Improves and Robustifies Class-Incremental Learning (2022, unknown): “The power of unlabeled data has recently drawn explosive interests, in multiple contexts such as semi-supervised learning (Chen et al., 2020d), self-training (Xie et al., 2020), and self-supervised learning (Chen et al., 2020e;c).”