Secure Evaluation of Quantized Neural Networks

By Assi Barak, Daniel E. Escudero, Anders Dalskov, Marcel Keller. 2019

159 citations recorded by the source corpus.

Abstract We investigate two questions in this paper: First, we ask to what extent “MPC friendly” models are already supported by major Machine Learning frameworks such as TensorFlow or PyTorch. Prior works provide protocols that only work on fixed-point integers and specialized activation functions, two aspects that are not supported by popular Machine Learning frameworks, and the need for these specialized model representations means that it is hard, and often impossible, to use e.g., TensorFlow to design, train and test models that later have to be evaluated securely. Second, we ask to what extent the functionality for evaluating Neural Networks already exists in general-purpose MPC frameworks. These frameworks have received more scrutiny, are better documented and supported on more platforms. Furthermore, they are typically flexible in terms of the threat model they support. In contrast, most secure evaluation protocols in the literature are targeted to a specific threat model and their implementations are only a “proof-of-concept”, making it very hard for their adoption in practice. We answer both of the above questions in a positive way:We observe that the quantization techniques supported by both TensorFlow, PyTorch and MXNet can provide models in a representation that can be evaluated securely; and moreover, that this evaluation can be performed by a general purpose MPC framework. We perform extensive benchmarks to understand the exact trade-offs between different corruption models, network sizes and efficiency. These experiments provide an interesting insight into cost between active and passive security, as well as honest and dishonest majority. Our work shows then that the separating line between existing ML frameworks and existing MPC protocols may be narrower than implicitly suggested by previous works.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Efficiency & Compression, Privacy & Security

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Efficiency & Compression

In Privacy & Security

Citations in the corpus

Advances and Open Problems in Federated Learning (2019, background): “As such a noticeable trend in research has consisted in designing custom protocols for applications such as linear and logistic regression [309, 172, 302] and neural network training and inference [302, 14, 46].”
CrypTFlow2: Practical 2-Party Secure Inference (2020, methodology): “The only known approaches to evaluate ReLUs efficiently require sacrificing security by making the untenable assumption that a non-colluding third party takes part in the protocol [7, 45, 50, 57, 62] or by leaking activations [12].”
MP-SPDZ: A Versatile Framework for Multi-Party Computation (2020, methodology): “The library provides functionality for logistic regression [54] and deep-learning inference [26].”
Cheetah: Lean and Fast Secure Two-Party Deep Neural Network Inference (2022, methodology): “As is shown, SecureQ8 was 3 × - 4 × faster than Cheetah on LAN, while Cheetah was 2 × - 3 × faster on WAN by the virtue of less communication 4 .”
CrypTFlow: Secure TensorFlow Inference (2019, methodology): “Even on MNIST/CIFAR, C RYP TF LOW has lower communication complexity and is more efﬁcient than prior and concurrent works [61], [44], [10].”
Falcon: Honest-Majority Maliciously Secure Framework for Private Deep Learning (2020, unknown): “QuantizedNN [14]* # G# G# # # # # # # # # G #”