Online Continual Learning through Mutual Information Maximization

By Yiduo Guo, B. Liu, Dongyan Zhao. 2022

147 citations recorded by the source corpus.

Subfields: Meta & Continual Learning, Information Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Meta & Continual Learning

1. Han-Jia Ye
2. De-chuan Zhan
3. Zixuan Ke
4. Da-Wei Zhou
5. Bin Liu
6. Liyuan Wang
7. Xingxing Zhang
8. Hang Su

In Information Theory

1. Jun Zhu
2. Gyuhak Kim
3. Bin Liu
4. Zixuan Ke

Citations in the corpus

• A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, methodology): “OCL InstAParam [68], CBRS [75], ASER [382], InfoRS [398], DVC [143], CTN [336], NCCL [475], ILOS [158], SCR [284], CVT [441], OCM [145], ER-ACE [47], RAR [498], PoLRS [50], AOP [144], CV [159]”
• Learnability and Algorithm for Continual Learning (2023, methodology): “…buffer data (Rusu et al., 2016; LopezPaz & Ranzato, 2017; Chaudhry et al., 2019a; Hou et al., 2019; Wu et al., 2019; Rolnick et al., 2019; Buzzega et al., 2020; Rajasegaran et al., 2020a; Liu et al., 2021; Cha et al., 2021; Yan et al., 2021; Wang et al., 2022b; Guo et al., 2022; Kim et al., 2022a).”
• A Theoretical Study on Solving Continual Learning (2022, background): “[40] Yiduo Guo, Bing Liu, and Dongyan Zhao.”
• Continual Training of Language Models for Few-Shot Learning (2022, methodology): “(2) Replay methods retain (Rebuffi et al., 2017; Lopez-Paz and Ranzato, 2017; Wang et al., 2020; Guo et al., 2022) or generate some data of old tasks (Shin et al.”
• Recent Advances of Continual Learning in Computer Vision: An Overview (2021, neutral): “We also give a brief discussion on some potential future research directions.”
• Class-Incremental Learning: A Survey (2023, methodology): “These exemplars are equally sampled from each seen class via the herding [124] algorithm in Definition 2.”
• Orchestrate Latent Expertise: Advancing Online Continual Learning with Multi-Level Supervision and Reverse Self-Distillation (2024, methodology): “For the two OCL benchmarks, we set memory size M = 1 k, 2 k, 5 k and M = 2 k, 4 k, 10 k , respectively (same as work [21, 22]).”
• A Multi-Head Model for Continual Learning via Out-of-Distribution Replay (2022, methodology): “There are many existing online CL systems, e.g., SLDA (Hayes & Kanan, 2020), REMIND (Hayes et al., 2020) and OCM (Guo et al., 2022).”
• Deep Class-Incremental Learning: A Survey (2023, background): “Future class-incremental learning algorithms should be able to conduct fully online training [226], [227], [228], [229], [230] without requiring the task boundaries [231], [232], [233].”
• Online Prototype Learning for Online Continual Learning (2023, methodology): “For base-lines, SCR [9], DVC [5], and OCM [6] also use a projection head to get representations.”