RMM: Reinforced Memory Management for Class-Incremental Learning

By Yaoyao Liu, Bernt Schiele, Qianru Sun. 2023

130 citations recorded by the source corpus.

Class-Incremental Learning (CIL) [40] trains classifiers under a strict memory budget: in each incremental phase, learning is done for new data, most of which is abandoned to free space for the next phase. The preserved data are exemplars used for replaying. However, existing methods use a static and ad hoc strategy for memory allocation, which is often sub-optimal. In this work, we propose a dynamic memory management strategy that is optimized for the incremental phases and different object classes. We call our method reinforced memory management (RMM), leveraging reinforcement learning. RMM training is not naturally compatible with CIL as the past, and future data are strictly non-accessible during the incremental phases. We solve this by training the policy function of RMM on pseudo CIL tasks, e.g., the tasks built on the data of the 0 -th phase, and then applying it to target tasks. RMM propagates two levels of actions: Level-1 determines how to split the memory between old and new classes, and Level-2 allocates memory for each specific class. In essence, it is an optimizable and general method for memory management that can be used in any replaying-based CIL method. For evaluation, we plug RMM into two top-performing baselines (LUCIR+AANets and POD+AANets [30]) and conduct experiments on three benchmarks (CIFAR-100, ImageNet-Subset, and ImageNet-Full). Our results show clear improvements, e.g., boosting POD+AANets by 3 . 6% , 4 . 4% , and 1 . 9% in the 25 -Phase settings of the above benchmarks, respectively. The code is available at https://class-il.mpi-inf.mpg.de/rmm

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Hang Su
5. Qiwen Wang
6. Jun Zhu
7. Ziwei Liu
8. Liyuan Wang

In Computer Vision and Pattern Recognition

1. Han-Jia Ye
2. De-chuan Zhan
3. Jun Zhu
4. Da-Wei Zhou
5. Hang Su
6. Liyuan Wang
7. Ziwei Liu
8. Qiwen Wang

In Reinforcement Learning

1. Liyuan Wang
2. Jun Zhu
3. De-chuan Zhan
4. Weiming Zhuang
5. Hang Su
6. Da-Wei Zhou
7. Ziwei Liu
8. Chen Chen

In Meta & Continual Learning

1. Han-Jia Ye
2. De-chuan Zhan
3. Da-Wei Zhou
4. Xingxing Zhang
5. Liyuan Wang
6. Hang Su
7. Qiwen Wang
8. Jun Zhu

Citations in the corpus

• A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, methodology): “Data Space iCaRL [356], GSS [17], Mnemonics [272], TPCIL [403], GDumb [340], DER++ [46], RMM [271], HAL [63], MRDC [431], CSI [217], X-DER [41]”
• GKEAL: Gaussian Kernel Embedded Analytic Learning for Few-Shot Class Incremental Task (2023, background): “The reinforcement memory management [18] handles the forgetting issue adopting reinforcement learning, and plugging it into PODNet and AANets leads to better performance.”
• A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental Learning (2022, background): “The research about CIL has attracted much interest [67, 32, 42] in the machine learning field.”
• Class-Incremental Learning: A Survey (2023, methodology): “We systematically choose 17 methods, including: Replay [128], RMM [144] ( data replay ), GEM [53] ( data regularization ), EWC [73] ( parameter regularization ), AANets [66], FOSTER [16], MEMO [17], DER [15], DyTox [18], L2P [69] ( dynamic networks ), LwF [81], iCaRL [82], PODNET [94], Coil [88] (…”
• Computationally Budgeted Continual Learning: What Does Matter? (2023, unknown): “Sa m pl in g TinyER [16] X FIFO,KMeans,Reservoir GSS [5] × GSS Random MIR [3] × Reservoir MIR GDumb [43] X Balanced Random MemOnly Mnemonics [34] × Mnemonics BalFineTune OCS [60] × OCS Random InfoRS [52] × MSE InfoRS Random RMM [33] × RMM ASER [51] × SV ASV RM [6] X Uncertainty Random AutoDA CLIB [3”
• TARGET: Federated Class-Continual Learning via Exemplar-Free Distillation (2023, methodology): “In CL, exemplar-based methods [44, 33, 54, 13] have achieved leading performance.”
• Balancing Between Forgetting and Acquisition in Incremental Subpopulation Learning (2022, methodology): “Given whether the training images can be retained, existing IL methods are divided into exemplar-based (EB) [5,24,13,11] [36,31,3,16] and non-exemplar-based (NEB) methods.”
• ACIL: Analytic Class-Incremental Learning with Absolute Memorization and Privacy Protection (2022, methodology): “%) are duplicated from [15]) except for the 3-combo method “POD+AANets+RMM” which is copied from the RMM paper [5] (its ImageNet-Full results are not listed due to no ImageNet option in the source code).”