Towards Probabilistic Verification of Machine Unlearning

By David Marco Sommer, Liwei Song, Sameer Wagh, Prateek Mittal. 2020

86 citations recorded by the source corpus.

Right to be forgotten, also known as the right to erasure, is the right of individuals to have their data erased from an entity storing it. The General Data Protection Regulation in the European Union legally solidified the status of this long held notion. As a consequence, there is a growing need for the development of mechanisms whereby users can verify if service providers comply with their deletion requests. In this work, we take the first step in proposing a formal framework to study the design of such verification mechanisms for data deletion requests -- also known as machine unlearning -- in the context of systems that provide machine learning as a service. We propose a backdoor-based verification mechanism and demonstrate its effectiveness in certifying data deletion with high confidence using the above framework. Our mechanism makes a novel use of backdoor attacks in ML as a basis for quantitatively inferring machine unlearning. In our mechanism, each user poisons part of its training data by injecting a user-specific backdoor trigger associated with a user-specific target label. The prediction of target labels on test samples with the backdoor trigger is then used as an indication of the user's data being used to train the ML model. We formalize the verification process as a hypothesis testing problem, and provide theoretical guarantees on the statistical power of the hypothesis test. We experimentally demonstrate that our approach has minimal effect on the machine learning service but provides high confidence verification of unlearning. We show that with a $30\%$ poison ratio and merely $20$ test queries, our verification mechanism has both false positive and false negative ratios below $10^{-5}$. Furthermore, we also show the effectiveness of our approach by testing it against an adaptive adversary that uses a state-of-the-art backdoor defense method.

Subfields: Artificial Intelligence, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

Citations in the corpus

Mixed-Privacy Forgetting in Deep Networks (2020, background): “Can we forget the cohort of interest without interfering with information about the other data and preserving, to the extent possible, the accuracy of the trained model? Recently, the problem of forgetting has received considerable attention [15, 16, 13, 19, 5, 24, 35, 41, 6, 39, 12, 7, 36], but sol”
A Survey of Machine Unlearning (2022, unknown): “Athena [139, 140] Python Model-agnostic https://github.”
DART: Data Addition and Removal Trees (2020, unknown): “DART models guarantee safeguards against these types of classical membership inference attacks and reduce the need for deletion-verification methods (Shintre and Dhaliwal 2019; Sommer et al. 2020); however, if an adversary has access to the updated model and the original model (the model trained on ”
Remember What You Want to Forget: Algorithms for Machine Unlearning (2021, background): “Other works in this space focus on exploring privacy risks (Chen et al., 2020) and veriﬁcation (Sommer et al., 2020) in machine unlearning settings.”
Machine Unlearning for Random Forests (2020, methodology): “DaRE models also reduce the need for deletion veriﬁcation methods (Shintre & Dhaliwal, 2019; Sommer et al., 2020).”
A Framework for Deprecating Datasets: Standardizing Documentation, Identification, and Communication (2021, background): “For instance, several approaches have been proposed in recent years for ‘machine unlearning’ 1, allowing data to be erased from already trained models [10, 14, 21, 93].”
Backdoor Learning: A Survey (2020, background): “CIFAR [86] 32× 32× 3 50,000 10,000 [44], [64], [13], [14], [47] [53], [60], [66], [67], [70] [15], [20], [21], [45], [51] [56], [61], [57], [63], [74] [84], [69], [9], [10], [25] [58] SVHN [87] 32× 32× 3 73,257 26,032 [51], [9], [10]”
Fast Yet Effective Machine Unlearning (2021, background): “[43] David Marco Sommer, Liwei Song, Sameer Wagh, and Prateek Mittal.”