Remembering for the Right Reasons: Explanations Reduce Catastrophic Forgetting

By Sayna Ebrahimi, Suzanne Petryk, Akash Gokul, W. Gan, Joseph E. Gonzalez, Marcus Rohrbach, Trevor Darrell. 2020

53 citations recorded by the source corpus.

The goal of continual learning (CL) is to learn a sequence of tasks without suffering from the phenomenon of catastrophic forgetting. Previous work has shown that leveraging memory in the form of a replay buffer can reduce performance degradation on prior tasks. We hypothesize that forgetting can be further reduced when the model is encouraged to remember the evidence for previously made decisions. As a first step towards exploring this hypothesis, we propose a simple novel training paradigm, called Remembering for the Right Reasons (RRR), that additionally stores visual model explanations for each example in the buffer and ensures the model has “the right reasons” for its predictions by encouraging its explanations to remain consistent with those used to make decisions at training time. Without this constraint, there is a drift in explanations and increase in forgetting as conventional continual learning algorithms learn new tasks. We demonstrate how RRR can be easily added to any memory or regularization-based approach and results in reduced forgetting, and more importantly, improved model explanations. We have evaluated our approach in the standard and few-shot settings and observed a consistent improvement across various CL approaches using different architectures and techniques to generate model explanations and demonstrated our approach showing a promising connection between explainability and continual learning. Our code is available at \url{https://github.com/SaynaEbrahimi/Remembering-for-the-Right-Reasons}

Subfields: Safety, Robustness & Interpretability, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

1. Hang Su
2. Jun Zhu
3. Yuxiang Bao
4. Hinrich Schütze
5. Gobinda Saha
6. Baochang Zhang
7. Runqi Wang
8. Álvaro Soto

In Meta & Continual Learning

1. Xingxing Zhang
2. Liyuan Wang
3. Hang Su
4. Bahram Zonooz
5. Elahe Arani
6. Jun Zhu
7. Prashant Bhat
8. Simone Calderara

Citations in the corpus

• A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, background): “Theauxiliaryinformationwithlowstoragecost,suchasclass statistics[79],[140]andattentionmaps[141],[142],canbe further incorporated to maintain old knowledge.”
• On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning (2022, neutral): “model responses [15, 12], explanations [25], etc.”
• Anti-Retroactive Interference for Lifelong Learning (2022, background): “RRR in [7] tries to save the correct attentions of previous images to avoid the attentions being affected by other tasks.”
• Recent Advances of Continual Learning in Computer Vision: An Overview (2021, neutral): “[95] stored both data instances and their model visual explanations (i.”
• Optimizing Reusable Knowledge for Continual Learning via Metalearning (2021, methodology): “Memory-based methods mitigate CF inserting data from past tasks into the training process of new tasks, continuously retraining previous tasks [30], either with raw samples [26, 31], or minimizing gradient interference [28, 32].”
• Continual Learning via Bit-Level Information Preserving (2021, methodology): “Replay Based Continual Learning These methods save representative examples of previous tasks in a replay buffer of limited size [19, 5, 13, 27, 37, 9] or train a generative model to generate samples of previous tasks [30, 31, 25, 32].”
• Task Agnostic Representation Consolidation: a Self-supervised based Continual Learning Approach (2022, background): “As the length of task sequence increases, our method clearly outperforms both RRR and ER.”
• BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Belief (2021, background): “While recent work in this area has focused on dynamic update of model parameters, e.g., (Ebrahimi et al., 2021), our work leaves the model fixed, and seeks improvement in the broader system in which the model is embedded, exploring an alternative and potentially more interpretable architecture…”
• Saliency Guided Experience Packing for Replay in Continual Learning (2021, background): “This demonstrate the benefit of saliency based input selection (for replay) in EPR over the saliency map regularization in RRR. Finally, we compare EPR with HAL [7] which holds the SOTA performance in this setup.”
• A Metaverse: Taxonomy, Components, Applications, and Open Challenges (2022, unknown): “Ebrahimi et al. [320] proposed a simple educational paradigm, Remembering for Right Reasons (RRR), by encouraging explanations so that models have the right reasons for their predictions.”
• CaSpeR: Latent Spectral Regularization for Continual Learning (2023, background): “, 2021) or providing richer replay signals (Buzzega et al., 2020a; Ebrahimi et al., 2021).”