Note on the quadratic penalties in elastic weight consolidation

By Ferenc Huszár. 2017

137 citations recorded by the source corpus.

Elastic weight consolidation (EWC, Kirkpatrick et al, 2017) is a novel algorithm designed to safeguard against catastrophic forgetting in neural networks. EWC can be seen as an approximation to Laplace propagation (Eskin et al, 2004), and this view is consistent with the motivation given by Kirkpatrick et al (2017). In this note, I present an extended derivation that covers the case when there are more than two tasks. I show that the quadratic penalties in EWC are inconsistent with this derivation and might lead to double-counting data from earlier tasks.

Subfields: Artificial Intelligence, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Meta & Continual Learning

Citations in the corpus

Toward Understanding Catastrophic Forgetting in Continual Learning (2019, methodology): “the Fisher information matrix computed from the previous task. Since the diagonal Fisher information approximates the average Hessian of the likelihoods, EWC is closely related to Laplace propagation [17, 43], where Laplace’s approximation is applied after each task to compute the regularizers. Besi”
Continuous Learning in Single-Incremental-Task Scenarios (2018, methodology): “A single set of optimal weights Θ is also maintained by using the most recent ones (Θ = Θ) since Θ already incorporates constraints from all previous batches (refer to the discussion in [7][13]).”
Progress & Compress: A scalable framework for continual learning (2018, methodology): “(Husz´ar, 2017) propose taking the expectation propagation (EP) (Minka, 2001) approach of keeping an explicit approximation term for each likelihood, so that when a task is revisited the approximation to its likelihood can be removed and recomputed.”
Continual World: A Robotic Benchmark For Continual Reinforcement Learning (2021, methodology): “We thus opt for a memory-efficiency solution presented in [21].”
Posterior Meta-Replay for Continual Learning (2021, methodology): “Online EWC.”
Lifelong Policy Gradient Learning of Factored Policies for Faster Training Without Forgetting (2020, background): “Huszár [17] noted that this does not correspond to the correct Bayesian formulation, and proposed to instead use − λ (cid:107) θ − α ( t − 1) (cid:107) 2 H ( t − 1) , where α ( t − 1) and H ( t − 1) capture all the information from tasks 1 through t − 1 in the Bayesian setting.”
Continual learning with hypernetworks (2019, methodology): “(Kirkpatrick et al., 2017; Huszár, 2018; Nguyen et al., 2018).”
Three scenarios for continual learning (2019, background): “It was pointed out by Huszár [33] that a slightly stricter 3 An alternative way to calculate ii would be, instead of taking for each training input x only the most likely label predicted by model p ˆ θ ( k ) , to sample for each x multiple labels from the entire conditional distribution deﬁned by…”
Online Structured Laplace Approximations For Overcoming Catastrophic Forgetting (2018, background): “This is more closely related to Laplace propagation, but may be overcounting early tasks [14] and does not approximate the posterior.”
Rotate your Networks: Better Weight Consolidation and Less Catastrophic Forgetting (2018, methodology): “Instead of using (3), we only use the FIM from the previous task because we found it works better, requires storage of only one FIM, and better fits the sequential Bayesian perspective (as shown in [18]).”
Memory Aware Synapses: Learning what (not) to forget (2017, methodology): “EWC uses individual penalty for each previous task, however, to make it computationally feasible we apply a single penalty as pointed out by [11].”
Overcoming Catastrophic Forgetting During Domain Adaptation of Seq2seq Language Generation (2022, methodology): “Existing work in LLL adopts the replay based methods (Pellegrini et al., 2019), such as Latent Replay, or regularization based methods (Huszár, 2018; Li and Hoiem, 2018), such as Elastic Weight Consolidation (EWC) (Kirkpatrick et al., 2017), Synaptic Intelligence (Zenke et al., 2017).”
Towards Robust Evaluations of Continual Learning (2018, methodology): “We train using the variant of EWC suggested in Husz´ar (2018).”
Generative replay with feedback connections as a general strategy for continual learning (2018, background): “- Online EWC: This is a modification of EWC proposed by [22], with inspiration from [23], that improves EWC’s scalability by ensuring the computational cost of the regularization term does not grow with number of tasks (see Appendix A.”
Self-Net: Lifelong Learning via Continual Self-Modeling (2018, methodology): “Note that training new task networks in this manner diﬀers from standard regularization methods (e.g., EWC; Kirkpatrick et al., 2017) because the weights learned for older tasks are not modiﬁed (and hence their performance does not degrade).”
Continual Contrastive Learning for Cross-Dataset Scene Classification (2022, methodology): “Kirkpatrick et al. [15] firstly proposed Elastic Weight Consolidation (EWC), an approach that employed a quadratic penalty term to constrain the update of important weights calculated through the diagonal of the Fisher information matrix.”
Efficient Continual Learning in Neural Networks with Embedding Regularization (2019, methodology): “g on more than two tasks, dierent penalties can be combined by storing the mean vectors and Fisher diagonal matrices for each task [14], or a single composite penalty can be obtained in various ways [19,11,15]. More in general, when faced with a sequential scenario, we would generate a sequence of w”
Continual Learning, Fast and Slow (2021, methodology): “First, we consider static architecture approaches of EWC [13] and O-EWC [89] (online EWC), which use a quadratic regularizer to penalize changes to important parameters of previous tasks according to the Fisher information.”
Continual Learning in Recurrent Neural Networks with Hypernetworks (2020, methodology): “We particularly focus on a version of EWC [11] called Online EWC, [12, 13] as an exemplar of weight-importance methods.”
Unifying Importance Based Regularisation Methods for Continual Learning (2022, methodology): “Regularisation based approaches introduced in [Kirkpatrick et al., 2017, Huszár, 2018] protect previous memories by modifying the loss function L k re-lated to dataset D k .”