Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Bounds

By Raef Bassily, Adam D. Smith, Abhradeep Thakurta. 2014

830 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Statistical Learning Theory, Privacy & Security

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistical Learning Theory

In Privacy & Security

Citations in the corpus

Deep Learning with Differential Privacy (2016, methodology): “, [53, 8]); we make several modifications and extensions, in particular in our privacy accounting.”
Membership Inference Attacks Against Machine Learning Models (2016, background): “Differential privacy [12] has been applied to linear and logistic regression [8], [37], support vector machines [28], risk minimization [5], [9], [34], deep learning [1], [30], learning an unknown probability distribution over a discrete population from random samples [10], and releasing hyper-param”
Practical and Private (Deep) Learning without Sampling or Shuffling (2021, methodology): “from D at each time step2 t, then privacy amplification by sampling (Kasiviswanathan et al., 2008; Bassily et al., 2014; Abadi et al., 2016; Wang et al., 2019) allows one to scale down the noise to σt · (k/n), while still ensuring εt-differential privacy.”
Privacy Amplification by Decentralization (2020, methodology): “An important application of this framework is Projected Noisy Stochastic Gradient Descent (PNSGD) in the centralized setting, where the trusted curator only reveals the ﬁnal model.”
Privacy Amplification via Random Check-Ins (2020, methodology): “We formally deﬁne the random check-ins procedure, describe a fully distributed DP-SGD protocol with random check-ins, and analyze its privacy and utility guarantees.”
Auditing Differentially Private Machine Learning: How Private is Private SGD? (2020, background): “The most prominent diﬀerentially private mechanism for training machine learning models is diﬀerentially private stochastic gradient descent (DP-SGD) [SCS13, BST14, ACG + 16].”
Differentially Private Learning with Adaptive Clipping (2019, neutral): “In example-level DP, datasets are considered neighbors when they differ by the addition or removal of a single example (Chaudhuri et al., 2011; Bassily et al., 2014; Abadi et al., 2016b; Papernot et al., 2017; Wu et al., 2017; Papernot et al., 2018; Iyengar et al., 2019).”
Subsampled Rényi Differential Privacy and Analytical Moments Accountant (2018, methodology): “…private machine learning models is to use Stochastic Gradient Descent (SGD) with diﬀerentially private releases of (sometimes clipped) gradients evaluated on mini-batches of a dataset (Song et al., 2013; Bassily et al., 2014; Wang et al., 2015; Foulds et al., 2016; Abadi et al., 2016).”
cpSGD: Communication-efficient and differentially-private distributed SGD (2018, background): “Providing rigorous privacy guarantees for machine learning applications has been an area of active recent interest [6, 35, 31].”
Ekiden: A Platform for Confidentiality-Preserving, Trustworthy, and Performant Smart Contracts (2018, positive): “We extended our machine learning contract with our own implementation of differentially-private stochastic gradient descent [13].”
Learning Anonymized Representations with Adversarial Neural Networks (2018, background): “In the machine learning literature, [Wasserman and Zhou, 2010] and [Chaudhuri et al., 2011] develop differentially private empirical risk minimization algorithms, and [Bassily et al., 2014] and [Wang et al., 2016] study similar statistical and sample complexity of differentially private procedures.”
Collecting Telemetry Data Privately (2017, background): “In the context of telemetry collection one typically considers algorithms that exhibit differential privacy in the local model [12, 14, 7, 5, 3, 18], also called randomized response model [19], γ-amplification [13], or FRAPP [1].”
Personalized and Private Peer-to-Peer Machine Learning (2017, background): “…privacy-preserving machine learning from centralized data, notably based on differential privacy (see Dwork and Roth, 2014; Chaudhuri et al., 2011; Bassily et al., 2014, and references therein), the case where sensitive datasets are distributed across multiple data owners has been much less…”
Federated Learning: Challenges, Methods, and Future Directions (2019, background): “Such sample-level privacy can be achieved in many learning tasks [2, 7, 22, 53, 85, 86].”