Machine unlearning: linear filtration for logit-based classifiers

By Thomas Baumhauer, Pascal Schöttle, Matthias Zeppelzauer. 2020

171 citations recorded by the source corpus.

Recently enacted legislation grants individuals certain rights to decide in what fashion their personal data may be used and in particular a “right to be forgotten”. This poses a challenge to machine learning: how to proceed when an individual retracts permission to use data which has been part of the training process of a model? From this question emerges the field of machine unlearning, which could be broadly described as the investigation of how to “delete training data from models”. Our work complements this direction of research for the specific setting of class-wide deletion requests for classification models (e.g. deep neural networks). As a first step, we propose linear filtration as an intuitive, computationally efficient sanitization method. Our experiments demonstrate benefits in an adversarial setting over naive deletion schemes.

Subfields: Artificial Intelligence, Safety Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety Research

Citations in the corpus

Mixed-Privacy Forgetting in Deep Networks (2020, methodology): “Can we forget the cohort of interest without interfering with information about the other data and preserving, to the extent possible, the accuracy of the trained model? Recently, the problem of forgetting has received considerable attention [15, 16, 13, 19, 5, 24, 35, 41, 6, 39, 12, 7, 36], but sol”
Towards Probabilistic Verification of Machine Unlearning (2020, methodology): “Second, the server can use our backdoor-based mechanism to quantitatively measure the effectiveness of recently proposed deletion approaches without strict deletion guarantees, such as [10, 11].”
Machine Unlearning for Random Forests (2020, background): “…K-means (Ginart et al., 2019), SVMs (Cauwenberghs & Poggio, 2001), logistic regression (Guo et al., 2020; Schelter, 2020), and deep neural networks (Baumhauer et al., 2020; Golatkar et al., 2020b; Wu et al., 2020); however, there is very limited work addressing the problem of efﬁcient data…”
A Framework for Deprecating Datasets: Standardizing Documentation, Identification, and Communication (2021, background): “For instance, several approaches have been proposed in recent years for ‘machine unlearning’ 1, allowing data to be erased from already trained models [10, 14, 21, 93].”
Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observations (2020, methodology): “[4] proposed a forgetting method for logit-based classification models by applying linear transformation to the output logits, but do not remove information from the weights.”