Secure Logistic Regression Based on Homomorphic Encryption: Design and Evaluation

By Miran Kim, Yongsoo Song, Shuang Wang, Yuhou Xia, Xiaoqian Jiang. 2018

296 citations recorded by the source corpus.

Background Learning a model without accessing raw data has been an intriguing idea to security and machine learning researchers for years. In an ideal setting, we want to encrypt sensitive data to store them on a commercial cloud and run certain analyses without ever decrypting the data to preserve privacy. Homomorphic encryption technique is a promising candidate for secure data outsourcing, but it is a very challenging task to support real-world machine learning tasks. Existing frameworks can only handle simplified cases with low-degree polynomials such as linear means classifier and linear discriminative analysis. Objective The goal of this study is to provide a practical support to the mainstream learning models (eg, logistic regression). Methods We adapted a novel homomorphic encryption scheme optimized for real numbers computation. We devised (1) the least squares approximation of the logistic function for accuracy and efficiency (ie, reduce computation cost) and (2) new packing and parallelization techniques. Results Using real-world datasets, we evaluated the performance of our model and demonstrated its feasibility in speed and memory consumption. For example, it took approximately 116 minutes to obtain the training model from the homomorphically encrypted Edinburgh dataset. In addition, it gives fairly accurate predictions on the testing dataset. Conclusions We present the first homomorphically encrypted logistic regression outsourcing model based on the critical observation that the precision loss of classification models is sufficiently small so that the decision plan stays still.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Wei Dai
2. Miran Kim
3. Hao Chen
4. Kim Laine
5. Yongsoo Song
6. Lichao Sun
7. Philip S. Yu
8. Lingjuan Lyu

Citations in the corpus

• Federated Machine Learning (2019, methodology): “In practice, additively homomorphic encryption [2] is widely used and polynomial approximations need to be made to evaluate non-linear functions in machine-learning algorithms, resulting in trade-offs between accuracy and privacy [4, 35].”
• Privacy-preserving logistic regression training (2018, result): “Independently and in parallel with our research, Kim et al. [5] investigated the same problem of performing the training phase of logistic regression in the encrypted domain.”
• Implementation and Performance Evaluation of RNS Variants of the BFV Homomorphic Encryption Scheme (2019, methodology): “Large multiplicative depths supported by our implementations are required for applications involving deep neural networks [19], logistic regression learning [20], and other important machine learning problems.”
• Privacy and Robustness in Federated Learning: Attacks and Defenses (2020, methodology): “Privacy-preserving techniques Existing Works Homomorphic Encryption [81], [82]”
• F1: A Fast and Programmable Accelerator for Fully Homomorphic Encryption (2021, background): “For instance, we have described an implementation of key-switching, but there are others [34, 45] with different tradeoffs.”
• Towards Deep Neural Network Training on Encrypted Data (2019, background): “Some prior work described how to preform training and inference for other types of models on encrypted data, specifically linear-regression models [24] and even logistic-regression models [29, 15, 19, 18, 10].”
• Secure Outsourced Matrix Computation and Application to Neural Networks (2018, unknown): “This technology has great potentials in many real-world applications such as statistical testing, machine learning, and neural networks [24, 30, 31, 40].”
• Efficient Multi-Key Homomorphic Encryption with Packed Ciphertexts with Application to Oblivious Neural Network Inference (2019, methodology): “Hence, HE schemeswith packing techniques [6, 7, 16, 22] have good amortized complexity per plaintext value, and they have been applied to privacy-preserving big data analysis [11, 37, 39].”
• Disparate Vulnerability: on the Unfairness of Privacy Attacks Against Machine Learning (2019, methodology): “As MIA just relies on the model outputs, it is effective even when techniques to learn on encrypted inputs are in place, e.g., homomorphic encryption [3, 13, 16] or multi-party computation [5, 20, 24].”
• Secure and Robust Machine Learning for Healthcare: A Survey (2020, methodology): “[114] Security Homomorphic encryption based secure logistic regression.”
• Logistic regression model training based on the approximate homomorphic encryption (2018, methodology): “A recent work [12] used the column-wise approach, i.”