A Differentially Private Stochastic Gradient Descent Algorithm for Multiparty Classification

By Arun Rajkumar, Shivani Agarwal. 2012

145 citations recorded by the source corpus.

Subfields: Federated & Distributed Learning, Privacy & Security

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Federated & Distributed Learning

In Privacy & Security

Citations in the corpus

Learning Privately over Distributed Features: An ADMM Sharing Approach (2019, background): “Pathak et al. (2010); Rajkumar and Agarwal (2012) apply DP to collaborative machine learning, with an inherent tradeoff between the privacy cost and utility achieved by the trained model.”
Personalized and Private Peer-to-Peer Machine Learning (2017, methodology): “Existing approaches for privacy-preserving distributed learning (see e.g., Pathak et al., 2010; Rajkumar and Agarwal, 2012; Shokri and Shmatikov, 2015; Huang et al., 2015) rely on a central (sometimes trusted) server, assume the local data distribution is the same for all users and/or are designed t”
Learning privately from multiparty data (2016, methodology): “In particular, several differentially-private algorithms were proposed, including parameter averaging through secure multiparty computation (Pathak et al., 2010), and private exchange of gradient information to minimize empirical risks incrementally (Rajkumar & Agarwal, 2012; Hamm et al., 2015).”
Differentially Private Distributed Online Learning (2018, background): “Due to [8], ; d ð Þ-differential privacy provides a slightlyweaker privacy guarantee than -differential privacy, but is more robust to the number of nodes.”
Distributed Learning without Distress: Privacy-Preserving Empirical Risk Minimization (2018, methodology): “All the above methods consume a total privacy budget of = 0.5, except Rajkumar and Agarwal which consumes = 0.5 budget each iteration.”
Stochastic gradient descent with differentially private updates (2013, methodology): “[10] use a differentially private algorithm based on a stochastic gradient descent based procedure for multiparty classification.”
Differentially Private Distributed Online Learning (2015, background): “Rajkumar and Agarwal [17] extended differentially private ERM classiﬁcation [8] to differentially private ERM multiparty classiﬁcation.”
Privacy-preserving Transfer Learning for Knowledge Sharing (2018, methodology): “A stochastic gradient descent among parties are used in (Rajkumar and Agarwal 2012), and then a multi-task learning method is proposed in (Xie et al. 2017).”
Differentially Private Hypothesis Transfer Learning (2018, background): “There is a large body of research work addressing this problem in a rigorous privacy-preserving manner, especially under the notion of differential privacy [34,14,36,40].”
Differentially Private Matrix Factorization (2015, background): “For the scend challenge, Rajkumar and Agarwal [2012] propose a smart solution by adding another noise vector ρs j (t) to each local gradient: ∇̃s vj (t) = ∇ is vj (t) + η is j + ρ is j (t).”
On the Intrinsic Privacy of Stochastic Gradient Descent (2019, background): “To address these concerns, privacy-enhanced stochastic gradient descent (SGD) have been proposed to train models [10], [20], [41], [47], [51].”
Secure, privacy-preserving and federated machine learning in medical imaging (2020, methodology): “DP can also be applied to algorithm updates during training, for instance in neural networks via differentially private stochastic gradient descent 59 or private aggregation of teacher ensembles 60 , or during inference time.”
AN EMPIRICAL STUDY (2022, unknown): “Private variants of learning algorithms have been proposed to address this need [8, 50, 42, 46, 17].”
FDML: A Collaborative Machine Learning Framework for Distributed Features (2019, background): “Relatively earlier, differential privacy has also been applied to collaborative machine learning [23, 24], with an inherent tradeoff between privacy and utility of the trained model.”
Secure and Efficient Federated Transfer Learning (2019, background): “Apart from this, guarding of revealed gradients in the current model with differential privacy [26] can be a considered as a solution.”