Privacy Aware Learning

By John C. Duchi, Michael I. Jordan, Martin J. Wainwright. 2012

299 citations recorded by the source corpus.

We study statistical risk minimization problems under a privacy model in which the data is kept confidential even from the learner. In this local privacy framework, we establish sharp upper and lower bounds on the convergence rates of statistical estimation procedures. As a consequence, we exhibit a precise tradeoff between the amount of privacy the data preserves and the utility, as measured by convergence rate, of any statistical estimator or learning procedure.

Subfields: Artificial Intelligence, Computer Networks and Communications, Privacy & Security

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Privacy & Security

Citations in the corpus

Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning (2017, background): “The authors introduce the concept of distributed deep learning as a way to protect the privacy of training data [83].”
Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data (2016, neutral): “Attempts to provide differential privacy for machine learning models led to a series of efforts on shallow machine learning models, including work by Bassily et al. (2014); Chaudhuri & Monteleoni (2009); Pathak et al. (2011); Song et al. (2013), and Wainwright et al. (2012).”
Breaking the Communication-Privacy-Accuracy Trilemma (2020, background): “Under an ε-LDP constraint, [63] gives a lower bound on the excess error for generalized linear or convex models:”
Federated Learning and Differential Privacy: Software tools analysis, the Sherpa.ai FL framework and methodological guidelines for preserving data privacy (2020, background): “On the other hand, in the case of a malicious or compromised server, or in the case of potential eavesdropping, DP can be applied on the local clients 3 for protecting their privacy [28, 29, 30].”
Personalized Federated Learning with Moreau Envelopes (2020, background): “Preserving privacy in FL has been studied in [21–25].”
Personalized Federated Learning: A Meta-Learning Approach (2020, background): “Several works have studied the problem of preserving privacy in federated learning (Duchi et al., 2014; McMahan et al., 2017b; Agarwal et al., 2018; Zhu et al., 2019).”
FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization (2019, background): “Another important challenge in federated learning is to preserve privacy in learning [Duchi et al., 2014].”
Personalized and Private Peer-to-Peer Machine Learning (2017, background): “local DP (Duchi et al., 2012; Kairouz et al., 2016), leads to very inaccurate models.”
Communication-Efficient Learning of Deep Networks from Decentralized Data (2016, background): “While federated learning offers many practical privacy benefits, providing stronger guarantees via differential privacy [14, 13, 1], secure multi-party computation [18], or their combination is an interesting direction for future work.”
Federated Learning: Challenges, Methods, and Future Directions (2019, background): “, gradient information, instead of the raw data [17, 31, 33].”
Crypto-Nets: Neural Networks over Encrypted Data (2014, background): “Another line of work focuses on differential privacy (Chaudhuri et al., 2011; Duchi et al., 2012; Dwork, 2008; Smith, 2011; Wasserman & Zhou, 2010).”
Federated Optimization: Distributed Machine Learning for On-Device Intelligence (2016, background): “Details of this are beyond the scope of the current work, but it is a promising direction for future research.”
LOGAN: Membership Inference Attacks Against Generative Models (2017, methodology): “vately trained models as the ﬁnal parameters are left unchanged. Differential Privacy [16] can be used to mitigate inference attacks, and it has been widely applied to various machine learning models [1,46,56,65,33,66]. Shokri and Shmatikov [56] support distributed training of deep learning networks”
LOGAN: Evaluating Privacy Leakage of Generative Models Using Generative Adversarial Networks (2017, background): “On the other hand, Differential Privacy [12] can mitigate inference attacks, and it has been widely applied to various machine learning models [1, 23, 30, 37, 42].”
Communication Complexity in Locally Private Distribution Estimation and Heavy Hitters (2019, background): “e publications of [1,2]. Private data release and computation has been studied in various domains, such as statistics, machine learning, database theory, algorithm design, and cryptography (See e.g., [3,4,5,6,7,8]). Diﬀerential Privacy (DP) [9] has emerged as one of the most popular notions of priva”
Stochastic, Distributed and Federated Optimization for Machine Learning (2017, background): “If protection from even a malicious (or compromised) coordinating server is needed, techniques from local differential privacy can be applied to privatize the individual updates [54].”