Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observations

By Aditya Golatkar, Alessandro Achille, Stefano Soatto. 2020

257 citations recorded by the source corpus.

We describe a procedure for removing dependency on a cohort of training data from a trained deep network that improves upon and generalizes previous methods to different readout functions and can be extended to ensure forgetting in the activations of the network. We introduce a new bound on how much information can be extracted per query about the forgotten cohort from a black-box network for which only the input-output behavior is observed. The proposed forgetting procedure has a deterministic part derived from the differential equations of a linearized version of the model, and a stochastic part that ensures information destruction by adding noise tailored to the geometry of the loss landscape. We exploit the connections between the activation and weight dynamics of a DNN inspired by Neural Tangent Kernels to compute the information in the activations.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Mixed-Privacy Forgetting in Deep Networks (2020, methodology): “[16] extend the framework of [15] to activations.”
DeltaGrad: Rapid retraining of machine learning models (2020, methodology): “Based on this defnition, (Golatkar et al., 2020a) and its follow-up work (Golatkar et al., 2020b) utilize the Fisher Information Matrix to approximately but effectively remove the information of the deleted training samples from general ML models.”
Unrolling SGD: Understanding Factors Influencing Machine Unlearning (2021, background): “[9], [13] is derived by adding a weighted term to the loss (to measure the KL divergence of the distribution of final weights), and minimizing the added term.”
Towards Unbounded Machine Unlearning (2023, methodology): “We use the same models with the same architectural modiﬁcations in [Golatkar et al., 2020a,b].”
When Machine Unlearning Jeopardizes Privacy (2020, background): “equires that, in addition to the data itself, any inﬂuence of the data on the model disappears [10,59]. This process, also called machine unlearning, has gained momentum both in academia and industry [7,9,10,17,18,20,27,35,50,55,62]. The most legit way to implement machine unlearning is to remove th”
DART: Data Addition and Removal Trees (2020, unknown): “…and Poggio 2001), logistic regression (Guo et al. 2020; Schelter 2020), and deep neural networks (Baumhauer, Sch ¨ ottle, and Zeppelza-uer 2020; Golatkar, Achille, and Soatto 2020b; Wu, Dobriban, and Davidson 2020) (for a comprehensive list of related work, see Section 6); however, there is…”
Machine Unlearning for Random Forests (2020, background): “…2019), SVMs (Cauwenberghs & Poggio, 2001), logistic regression (Guo et al., 2020; Schelter, 2020), and deep neural networks (Baumhauer et al., 2020; Golatkar et al., 2020b; Wu et al., 2020); however, there is very limited work addressing the problem of efﬁcient data deletion for tree-based models…”
TOFU: A Task of Fictitious Unlearning for LLMs (2024, unknown): “While this is a promising direction for products and services that wrap API-based models, it amounts to a form of prompt engineering and does not yield any real privacy in terms of the Right to be Forgotten.”
LQF: Linear Quadratic Fine-Tuning (2020, methodology): “We leave various applications of LQF to future work, including its use in (i) uncertainty quantification, (ii) backward compatibility [44], (iii) leave-one-out cross validation without a validation set [9], (iv) active/online learning [45], (v) counterfactual analysis in the first order, (vi) contin”
The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning (2024, background): “…as a response to privacy concerns in light of regulation (Council of European Union, 2014; CCPA, 2018), and most methods focused on erasing specific samples or facts (Golatkar et al., 2020; Liu et al., 2020; Meng et al., 2022; Jang et al., 2023; Pawelczyk et al., 2023) rather than entire domains.”
Fast Yet Effective Machine Unlearning (2021, methodology): “Initial work on forgetting in convolutional networks is presented in [19,20].”