Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks
By Aditya Golatkar, Alessandro Achille, Stefano Soatto. 2019
845 citations recorded by the source corpus.
We explore the problem of selectively forgetting a particular subset of the data used for training a deep neural network. While the effects of the data to be forgotten can be hidden from the output of the network, insights may still be gleaned by probing deep into its weights. We propose a method for "scrubbing" the weights clean of information about a particular set of training data. The method does not require retraining from scratch, nor access to the data originally used for training. Instead, the weights are modified so that any probing function of the weights is indistinguishable from the same function applied to the weights of a network trained without the data to be forgotten. This condition is a generalized and weaker form of Differential Privacy. Exploiting ideas related to the stability of stochastic gradient descent, we introduce an upper-bound on the amount of information remaining in the weights, which can be estimated efficiently even for deep neural networks.
Subfields: Artificial Intelligence, Meta & Continual Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Percy Liang
- Mohit Bansal
- Sergey Levine
- Luke S. Zettlemoyer
- Yoshua Bengio
- Nicholas Carlini
- Jianfeng Gao
- Quoc V. Le
In Meta & Continual Learning
- Stefano Soatto
- Alessandro Achille
- Aditya Golatkar
- Avinash Ravichandran
- Marzia Polito
- Peter Stone
- Bo Liu
- Qian Liu
Citations in the corpus
- A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, neutral): “…oftrainingsamples[307],utilizinghistoricalparametersand gradients[308],removingoldknowledgefromparameterswith Fisherinformationmatrix[309],addingadaptiveparameterson apre-trainedmodel[310],etc. Ontheotherhand,continual learningmaysufferfromdataleakageandprivacyinvasionas…”
- Learning with Selective Forgetting (2021, methodology): “Inspired by differential privacy [Abadi et al. , 2016], Eternal Sunshine of the Spotless Net [Golatkar et al. , 2020] introduced a scrubbing procedure that removes information from the trained weights of deep neural networks using the Fisher information matrix.”
- Mixed-Privacy Forgetting in Deep Networks (2020, methodology): “[16] extend the framework of [15] to activations.”
- DeltaGrad: Rapid retraining of machine learning models (2020, methodology): “Starting from the concept of Differential Privacy, (Golatkar et al., 2020a) proposes a defnition on how much information about the training samples to be forgotten may remain in the weights of the network after the removal of those samples.”
- Machine Unlearning (2019, background): “This includes two concurrent proposals [34], [35] The mechanism by Guo et al.”
- Measuring Forgetting of Memorized Training Examples (2022, background): “Algorithms for machine unlearning have been proposed for k -means [26], empirical risk minimization [31, 37, 59, 77, 68], and deep learning [20, 28, 27, 60, 5].”
- Analyzing Information Leakage of Updates to Natural Language Models (2019, background): “Contemporary approximate deletion methods [15, 16] yield models that are only statistically indistinguishable from a model retrained from scratch.”
- TOFU: A Task of Fictitious Unlearning for LLMs (2024, positive): “Question answering Some prior work focuses on classification models (e.g Guo et al., 2019; Golatkar et al., 2020; Kurmanji et al., 2023a; Wang et al., 2023; Chen & Yang, 2023; Pawelczyk et al., 2023), but with recent advancements in chatbots and instruction-tuned LLMs, we need to shift our…”
- Unrolling SGD: Understanding Factors Influencing Machine Unlearning (2021, background): “Alternatively various approaches suggest hessian-based updates [11], [9].”
- A Survey of Machine Unlearning (2022, positive): “overtrained deep neural networks by actively unlearning useless, obsolete, or redundant data samples that could cause catastrophic forgetting [35, 53].”
- Machine Unlearning for Random Forests (2020, background): “…2019), SVMs (Cauwenberghs & Poggio, 2001), logistic regression (Guo et al., 2020; Schelter, 2020), and deep neural networks (Baumhauer et al., 2020; Golatkar et al., 2020b; Wu et al., 2020); however, there is very limited work addressing the problem of efficient data deletion for tree-based models…”
- Towards Unbounded Machine Unlearning (2023, methodology): “We use the same models with the same architectural modifications in [Golatkar et al., 2020a,b].”
- Rethinking machine unlearning for large language models (2024, neutral): “To address these challenges, many research efforts have shifted towards the development of scalable and effective approximate unlearning methods Golatkar et al. (2020) Challenges of MU for LLMs LLM unlearning introduces new challenges and complexities.”
- Machine unlearning: linear filtration for logit-based classifiers (2020, methodology): “On the other hand our method’s intuitive simplicity nicely complements concurrent approaches such as [14], [17], [18].”
- Fast Yet Effective Machine Unlearning (2021, methodology): “Similarly, although NegGrad seems to unlearn the forget classes properly, but its performance on Dr takes a hit (22% accuracy retained vs 80.21% retained by our method).”
- Continual Learning and Private Unlearning (2022, methodology): “Based on differential privacy, Golatkar et al. introduced “scrubbing” that removes information from the weights of deep networks based on the Fisher Information Matrix (Golatkar et al., 2020).”
- Editing Models with Task Arithmetic (2022, neutral): “[34] Aditya Golatkar, Alessandro Achille, and Stefano Soatto.”
- Knowledge Unlearning for Mitigating Privacy Risks in Language Models (2022, background): “Several studies attempt to explore machine unlearning for deep neural networks (Golatkar et al., 2020; Mehta et al., 2022).”
- Machine Unlearning of Features and Labels (2021, background): “[31, 32] move to the field of privacy and use influence functions to “forget” data points in neural networks using special approximations.”