TRGP: Trust Region Gradient Projection for Continual Learning

By Sen Lin, Li Yang, Deliang Fan, Junshan Zhang. 2022

116 citations recorded by the source corpus.

Catastrophic forgetting is one of the major challenges in continual learning. To address this issue, some existing methods put restrictive constraints on the optimization space of the new task for minimizing the interference to old tasks. However, this may lead to unsatisfactory performance for the new task, especially when the new task is strongly correlated with old tasks. To tackle this challenge, we propose Trust Region Gradient Projection (TRGP) for continual learning to facilitate the forward knowledge transfer based on an efficient characterization of task correlation. Particularly, we introduce a notion of `trust region' to select the most related old tasks for the new task in a layer-wise and single-shot manner, using the norm of gradient projection onto the subspace spanned by task inputs. Then, a scaled weight projection is proposed to cleverly reuse the frozen weights of the selected old tasks in the trust region through a layer-wise scaling matrix. By jointly optimizing the scaling matrices and the model, where the model is updated along the directions orthogonal to the subspaces of old tasks, TRGP can effectively prompt knowledge transfer without forgetting. Extensive experiments show that our approach achieves significant improvement over related state-of-the-art methods.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Optimization & Training, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Hang Su
2. Jun Zhu
3. Liyuan Wang
4. Xingxing Zhang
5. René Vidal
6. Paris V. Giampouras
7. Liangzu Peng
8. Dacheng Tao

In Computer Vision and Pattern Recognition

1. Dacheng Tao
2. Jun Zhu
3. Hang Su
4. Liyuan Wang
5. Xingxing Zhang
6. René Vidal
7. Paris V. Giampouras
8. Liangzu Peng

In Optimization & Training

1. Yajing Kong
2. Liyuan Wang
3. Dacheng Tao
4. René Vidal
5. Jun Zhu
6. Sen Lin
7. Hang Su
8. Liu Liu

In Meta & Continual Learning

1. Xingxing Zhang
2. Liyuan Wang
3. Hang Su
4. Jun Zhu
5. René Vidal
6. Paris V. Giampouras
7. Liangzu Peng
8. Yajing Kong

Citations in the corpus

• A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “TRGP [178] defines the “trust region” through the norm of gradient projection onto the subspace of previous inputs, so as to selectively reuse the frozen weights of old tasks.”
• The Ideal Continual Learner: An Agent That Never Forgets (2023, background): “Put differently, the ability of a learner to improve the performance on previous tasks with knowledge from new tasks is desired (only) when previous tasks are solved sub-optimally; such ability is typically called positive backward transfer (Lin et al., 2022b).”
• Beyond Not-Forgetting: Continual Learning with Backward Knowledge Transfer (2022, methodology): “It can be seen from Table 3 that CUBER performs even better than TRGP although CUBER follows the same strategy, i.e., the scaled weight projection, to facilitate the forward knowledge transfer, and achieves the best FWT among the three methods in most cases.”
• Task-Free Continual Learning via Online Discrepancy Distance Learning (2022, neutral): “One of the most popular approaches to relieve forgetting is by imposing a regularization loss within the optimization procedure [7, 11, 13, 16, 19, 25, 26, 31, 34, 35, 40, 41, 57], where the network’s parameters which are important to the past tasks are penalized when updating.”
• Class Gradient Projection For Continual Learning (2022, methodology): “In the construction for task 𝜏 > 1, we construct the common bases for class 𝑗 and 𝑘 as in [26].”
• Balancing Stability and Plasticity through Advanced Null Space in Continual Learning (2022, methodology): “Algorithms-based methods design the update rule to decrease the interference of parameter update on the performance of old tasks [6,7,24,26,43,47,49,54].”
• Hebbian Learning based Orthogonal Projection for Continual Learning of Spiking Neural Networks (2024, background): “Lin et al. (2022) improve Saha et al. (2021) with trust region projection considering similarity between tasks.”