Robust Traceability from Trace Amounts

By Cynthia Dwork, Adam D. Smith, Thomas Steinke, Jonathan Ullman, Salil P. Vadhan. 2015

192 citations recorded by the source corpus.

The privacy risks inherent in the release of a large number of summary statistics were illustrated by Homer et al. (PLoS Genetics, 2008), who considered the case of 1-way marginals of SNP allele frequencies obtained in a genome-wide association study: Given a large number of minor allele frequencies from a case group of individuals diagnosed with a particular disease, together with the genomic data of a single target individual and statistics from a sizable reference dataset independently drawn from the same population, an attacker can determine with high confidence whether or not the target is in the case group. In this work we describe and analyze a simple attack that succeeds even if the summary statistics are significantly distorted, whether due to measurement error or noise intentionally introduced to protect privacy. Our attack only requires that the vector of distorted summary statistics is close to the vector of true marginals in ℓ1 norm. Moreover, the reference pool required by previous attacks can be replaced by a single sample drawn from the underlying population. The new attack, which is not specific to genomics and which handles Gaussian as well as Bernouilli data, significantly generalizes recent lower bounds on the noise needed to ensure differential privacy (Bun, Ullman, and Vadhan, STOC 2014, Steinke and Ullman, 2015), obviating the need for the attacker to control the exact distribution of the data.

Subfields: Artificial Intelligence, Computer Science Applications, Information Systems, Computational Chemistry & Drug Discovery

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Science Applications

In Information Systems

In Computational Chemistry & Drug Discovery

Citations in the corpus

The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation (2021, background): “…individual client updates, the server is still able to learn the sum of the updates, which itself may leak potentially sensitive information (Melis et al., 2019; Carlini et al., 2019; Song & Shmatikov, 2019a; Dwork et al., 2015; Song & Shmatikov, 2019b; Nasr et al., 2021; Shokri et al., 2017).”
The structure of optimal private tests for simple hypotheses (2018, background): “[15, 35]), and we remain far from a general characterization of the statistical cost of privacy.”
Tight Lower Bounds for Differentially Private Selection (2017, methodology): “To illustrate the versatility of Theorem 3, we show how it implies known lower bounds for releasing the mean of the dataset [16], [17], [18].”
Machine Learning Models that Remember Too Much (2017, background): “Subsequent work extended this work to published noisy statistics [24] and MicroRNA-based studies [5].”
Auditing Data Provenance in Text-Generation Models (2018, background): “t is a member of D. Membership inference attacks against aggregate statistics have been demonstrated in the context of genomic studies [13], location time-series [26], and noisy statistics in general [8]. Shokri et al. [28] develop black-box membership inference techniques against ML models which pe”
Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning (2018, background): “In a membership inference attack, however, the attacker’s objective is to infer if a particular individual data record was included in the training dataset [4], [5], [6].”
Machine Learning with Membership Privacy using Adversarial Regularization (2018, methodology): “ata. The data required for training accurate models presents serious privacy issues. The leakage through complex machine learning models maybe less obvious, compared to, for example, linear statistics[18].However,machine learning models,similartoother types of computations, could signiﬁcantly leak i”
Measuring Forgetting of Memorized Training Examples (2022, methodology): “Membership Inference In membership inference [31, 19, 80, 68], an adversary infers whether or not a target example was contained in a model’s training set.”
Membership Inference Attacks From First Principles (2021, methodology): “First explored as tracing attacks [11, 12, 22, 59] on medical datasets, they were exended to machine learning models as membership inference attacks [60].”
Exposed! A Survey of Attacks on Private Data (2017, methodology): “The specific technical conditions we need to impose are beyond the scope of this article but can be found in Dwork et al. (2015d).”