QUOTIENT: Two-Party Secure Neural Network Training and Prediction

By Nitin Agrawal, Ali Shahin Shamsabadi, Matt J. Kusner, Adrià Gascón. 2019

245 citations recorded by the source corpus.

Recently, there has been a wealth of effort devoted to the design of secure protocols for machine learning tasks. Much of this is aimed at enabling secure prediction from highly-accurate Deep Neural Networks (DNNs). However, as DNNs are trained on data, a key question is how such models can be also trained securely. The few prior works on secure DNN training have focused either on designing custom protocols for existing training algorithms, or on developing tailored training algorithms and then applying generic secure protocols. In this work, we investigate the advantages of designing training algorithms alongside a novel secure protocol, incorporating optimizations on both fronts. We present QUOTIENT, a new method for discretized training of DNNs, along with a customized secure two-party protocol for it. QUOTIENT incorporates key components of state-of-the-art DNN training such as layer normalization and adaptive gradient methods, and improves upon the state-of-the-art in DNN training in two-party computation. Compared to prior work, we obtain an improvement of 50X in WAN time and 6% in absolute accuracy.

Subfields: Artificial Intelligence, Privacy & Security

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Privacy & Security

Citations in the corpus

Advances and Open Problems in Federated Learning (2019, methodology): “Similar protections can be achieved under the MPC model [302, 14].”
Privacy in Deep Learning: A Survey (2020, methodology): “propose QUOTIENT [93] where their goal is to design an optimization algorithm alongside a secure computation protocol customized for it, instead of a conventional approach which is using encryption on top of existing optimization algorithms.”
Cheetah: Lean and Fast Secure Two-Party Deep Neural Network Inference (2022, methodology): “Cheetah contributes a set of novel cryptographic protocols for the most common linear operations and non-linear operations of DNNs. Cheetah can perform secure inference on large models, e.g. ResNet, and DenseNet [28], with signiﬁcantly smaller computation and communication overheads than the state-o”
Falcon: Honest-Majority Maliciously Secure Framework for Private Deep Learning (2020, methodology): “EPIC [52] demonstrates the use of transfer learning in the space of privacy-preserving machine learning while Quotient [53] takes the first steps in developing two party secure computation protocols for optimizers and normalizations.”
Trustworthy AI: From Principles to Practices (2021, methodology): “To reduce the high costs of computation and communication of the SMPC, parameter quantization and function approximation were used together with cryptographic protocols in References [8, 32].”
CrypTFlow: Secure TensorFlow Inference (2019, background): “…of ﬁxed-point for secure computations is well-known [51], prior works on secure inference have addressed the ﬂoat-to-ﬁxed problem by either generating a ﬁxed-point model by hand ([45], [42], [38], [44], [61]), or by using non-standard training algorithms that output ﬁxed-point models ([54], [5]).”