SecureNN: 3-Party Secure Computation for Neural Network Training

By Sameer Wagh, Divya Gupta, Nishanth Chandran. 2019

486 citations recorded by the source corpus.

Abstract Neural Networks (NN) provide a powerful method for machine learning training and inference. To effectively train, it is desirable for multiple parties to combine their data – however, doing so conflicts with data privacy. In this work, we provide novel three-party secure computation protocols for various NN building blocks such as matrix multiplication, convolutions, Rectified Linear Units, Maxpool, normalization and so on. This enables us to construct three-party secure protocols for training and inference of several NN architectures such that no single party learns any information about the data. Experimentally, we implement our system over Amazon EC2 servers in different settings. Our work advances the state-of-the-art of secure computation for neural networks in three ways: 1. Scalability: We are the first work to provide neural network training on Convolutional Neural Networks (CNNs) that have an accuracy of > 99% on the MNIST dataset; 2. Performance: For secure inference, our system outperforms prior 2 and 3-server works (SecureML, MiniONN, Chameleon, Gazelle) by 6×-113× (with larger gains obtained in more complex networks). Our total execution times are 2 − 4× faster than even just the online times of these works. For secure training, compared to the only prior work (SecureML) that considered a much smaller fully connected network, our protocols are 79× and 7× faster than their 2 and 3-server protocols. In the WAN setting, these improvements are more dramatic and we obtain an improvement of 553×! 3. Security: Our protocols provide two kinds of security: full security (privacy and correctness) against one semi-honest corruption and the notion of privacy against one malicious corruption [Araki et al. CCS’16]. All prior works only provide semi-honest security and ours is the first system to provide any security against malicious adversaries for the secure computation of complex algorithms such as neural network inference and training. Our gains come from a significant improvement in communication through the elimination of expensive garbled circuits and oblivious transfer protocols.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Privacy in Deep Learning: A Survey (2020, background): “onw/DNN Private Collaborative NN [102] # MNIST ImageClassiﬁcationw/DNN Secure Aggregation for ML [103] # # - Federated Learning QUOTIENT [104] # # MNIST, Thyroid, Credit Classiﬁcation w/ DNN SecureNN [105] # # MNIST ImageClassiﬁcationw/DNN ABY3 [106] # # MNIST LIR, LOR, NN SecureML [107] # MNIST, Gi”
Secure Evaluation of Quantized Neural Networks (2019, methodology): “SecureNN [63] can be seen as an extension of SecureML where both three-and four-party protocols (both with one corrupted party) are used.”
QUOTIENT: Two-Party Secure Neural Network Training and Prediction (2019, methodology): “While there have been massive practical developments in both cryptography and ML (including the works above), most recent works for model training [22, 40, 41, 54] and prediction [7, 15, 50] on encrypted data are largely based on optimizations for either the ML model or the employed cryptographic te”
End-to-end privacy preserving deep learning on multi-institutional medical imaging (2021, methodology): “Unlike classical data secret sharing schemes like SecureNN 29 , where a shared input [[ x ]] is applied on a public function f , FSS applies a public input x on a private shared function Both approaches output a secret shared result.”
Falcon: Honest-Majority Maliciously Secure Framework for Private Deep Learning (2020, methodology): “29 SecureML [4]* Semi-honest WAN 4336 SecureNN [12] Semi-honest WAN 7.”
DarKnight: A Data Privacy Scheme for Training and Inference of Deep Neural Networks (2020, methodology): “Secure multi-party computing is another approach, where multiple non-colluding servers may use custom data exchange protocols to protect input data [33, 24, 28, 29].”
A Scalable Approach for Privacy-Preserving Collaborative Machine Learning (2020, background): “While several more recent works have considered MPC-based learning setups with information-theoretic privacy [37, 27], their constructions are limited to three or four parties.”