The Value of Collaboration in Convex Machine Learning with Differential Privacy

By Nan Wu, Farhad Farokhi, David B. Smith, Mohamed Ali Kâafar. 2019

112 citations recorded by the source corpus.

In this paper, we apply machine learning to distributed private data owned by multiple data owners, entities with access to non-overlapping training datasets. We use noisy, differentially-private gradients to minimize the fitness cost of the machine learning model using stochastic gradient descent. We quantify the quality of the trained model, using the fitness cost, as a function of privacy budget and size of the distributed datasets to capture the trade-off between privacy and utility in machine learning. This way, we can predict the outcome of collaboration among privacy-aware data owners prior to executing potentially computationally-expensive machine learning algorithms. Particularly, we show that the difference between the fitness of the trained machine learning model using differentially-private gradient queries and the fitness of the trained machine model in the absence of any privacy concerns is inversely proportional to the size of the training datasets squared and the privacy budget squared. We successfully validate the performance prediction with the actual performance of the proposed privacy-aware learning algorithms, applied to: financial datasets for determining interest rates of loans using regression; and detecting credit card frauds using support vector machines.

Subfields: Artificial Intelligence, Federated & Distributed Learning, Privacy & Security

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. H. Vincent Poor
2. Ming Ding
3. Chuan Ma
4. Jun Li
5. Farhad Farokhi
6. Shi Jin
7. Tony Q. S. Quek
8. Bo Liu

In Federated & Distributed Learning

1. H. Vincent Poor
2. Ming Ding
3. Mingzhe Chen
4. Walid Saad

In Privacy & Security

1. Chuan Ma
2. Jun Li
3. Ming Ding
4. Farhad Farokhi
5. H. Vincent Poor
6. Tony Q. S. Quek
7. Bo Liu
8. Zihuai Lin

Citations in the corpus

• Federated Learning With Differential Privacy: Algorithms and Performance Analysis (2019, background): “The work in [30] proposed novel DP based SGD algorithms and analyzed their performance bounds which were shown to be related to privacy levels and the sizes of datasets.”
• When Machine Learning Meets Privacy (2020, background): “e usually not shared among users. Moreover, different privacy protection schemes in centralised learning, such as encryption, perturbation, can be easily extended to the distributed learning settings [169]. In this sense, private ML in distributed systems have a lot in common with that of centralise”