Online Class-Incremental Continual Learning with Adversarial Shapley Value

By Dongsub Shim, Zheda Mai, Jihwan Jeong, Scott Sanner, Hyunwoo J. Kim, Jongseong Jang. 2020

267 citations recorded by the source corpus.

As image-based deep learning becomes pervasive on every device, from cell phones to smart watches, there is a growing need to develop methods that continually learn from data while minimizing memory footprint and power consumption. While memory replay techniques have shown exceptional promise for this task of continual learning, the best method for selecting which buffered images to replay is still an open question. In this paper, we specifically focus on the online class-incremental setting where a model needs to learn new classes continually from an online data stream. To this end, we contribute a novel Adversarial Shapley value scoring method that scores memory data samples according to their ability to preserve latent decision boundaries for previously observed classes (to maintain learning stability and avoid forgetting) while interfering with latent decision boundaries of current classes being learned (to encourage plasticity and optimal learning of new class boundaries). Overall, we observe that our proposed ASER method provides competitive or improved performance compared to state-of-the-art replay-based continual learning methods on a variety of datasets.

Subfields: Artificial Intelligence, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Meta & Continual Learning

Citations in the corpus

A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “More advanced strategies prioritize the replay of those training samples that are informative [273], diversiﬁed in parameter gradients [21], balanced in class labels [159], [274], and beneﬁcial to latent decision boundaries [134].”
Continual Learning by Modeling Intra-Class Variation (2022, methodology): “ASER (Shim et al., 2021) leverages Shapley value (Roth, 1988) to select buffer data.”
Repeated Augmented Rehearsal: A Simple but Strong Baseline for Online Continual Learning (2022, methodology): “As shown in Table 1, RAR improves ER and ER variants (MIR, ASER and SCR).”
Online Continual Learning with Contrastive Vision Transformer (2022, methodology): “This paper focuses on a more realistic and challenging setting: online continual learning (online CL) [34,50,67], where the model learns a sequence of classification tasks with a single pass over the data and without task boundaries.”
Adaptive Orthogonal Projection for Batch and Online Continual Learning (2022, neutral): “ThecodeforASER/AGEM/GDumb: https://github.com/RaptorMai/ online-continual-learning.”
Representation Compensation Networks for Continual Semantic Segmentation (2022, background): “The knowledge can be stored by multiple types, like examples [4, 6, 9, 11, 73, 83], prototypes [35, 106, 107], generative networks [60], etc.”
Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning System (2022, background): “There are a plethora of evaluation protocols in the CL literature, each of which biases the evaluation towards a certain approach (Farquhar & Gal, 2018; Mi et al., 2020; Shim et al., 2020; van de Ven & Tolias, 2019).”
GCR: Gradient Coreset based Replay Buffer Selection for Continual Learning (2021, neutral): “Coresets for Replay-based CL: Approaches include selecting replay buffers by maximizing criteria like sample gradient diversity ( GSS [8]), a mix of mini-batch gradient similarity and cross-batch diversity ( OCS [9]), adversarial Shapley value( ASER [40]), or representation matching (iCARL [22]).”
Online Continual Learning Via Candidates Voting (2021, methodology): “In Section 4.2, we compare our online method with existing online work including A-GEM [8], MIR [2], GSS [3], ASER [37] and GDUMB [34].”
Recent Advances of Continual Learning in Computer Vision: An Overview (2021, positive): “There are also a few other methods [13, 88] that focused on selecting data instances from the memory buffer for retraining.”
New Insights on Reducing Abrupt Representation Change in Online Continual Learning (2021, methodology): “Our evaluation includes the metrics and experimental settings used in previous works on online continual learning with a single-head (Aljundi et al., 2019a; Ji et al., 2020; Shim et al., 2020).”
Supervised Contrastive Replay: Revisiting the Nearest Class Mean Classifier in Online Class-Incremental Continual Learning (2021, methodology): “For instance, in CIFAR100, the NCM classifier helps ASERµ with 1k memory achieve 22%, which requires five times more memory to achieve when using the Softmax classifier.”
Online Prototype Learning for Online Continual Learning (2023, methodology): “To explore the effects of Rot, we employ it for some SOTA baselines, as shown in Table S5.”
One Size Fits All for Semantic Shifts: Adaptive Prompt Tuning for Continual Learning (2023, background): “…which rely on retaining and replaying subsets of past data (Rolnick et al., 2019; Bang et al., 2021; Buzzega et al., 2020; Aljundi et al., 2019a; Shim et al., 2021; Aljundi et al., 2019b; Prabhu et al., 2020; Koh et al., 2022; Yoon et al., 2023; Kim et al., 2024); regularization techniques,…”
Continual Semantic Segmentation with Automatic Memory Sample Selection (2023, methodology): “Most the previous selection methods rely on manually-designed strategies based on heuristic rules such as sample diversity [1, 4, 26, 49], adversarial Shapley value [40] or feature matching [38].”
Resurrecting Old Classes with New Data for Exemplar-Free Continual Learning (2024, methodology): “ASER [43] used the kNN-specific Shapley value to obtain more representative buffer samples.”
Online Continual Learning For Interactive Instruction Following Agents (2024, background): “In alignment with the common practice in the prior arts (Aljundi et al., 2019a; Buzzega et al., 2020; Shim et al., 2021), we assume that both setups follow an online and disjoint paradigm.”
Retrospective Adversarial Replay for Continual Learning (2022, methodology): “However, selecting such samples [2, 55] from a small buffer only brings limited improvement because the buffer (often formed using reservoir sampling [62] and thus is uniform) does not well cover many such marginal samples.”