Faster CryptoNets: Leveraging Sparsity for Real-World Encrypted Inference
By Edward Chou, Josh Beal, Daniel Levy, Serena Yeung, Albert Haque, Li Fei-Fei. 2018
228 citations recorded by the source corpus.
Homomorphic encryption enables arbitrary computation over data while it remains encrypted. This privacy-preserving feature is attractive for machine learning, but requires significant computational time due to the large overhead of the encryption scheme. We present Faster CryptoNets, a method for efficient encrypted inference using neural networks. We develop a pruning and quantization approach that leverages sparse representations in the underlying cryptosystem to accelerate inference. We derive an optimal approximation for popular activation functions that achieves maximally-sparse encodings and minimizes approximation error. We also show how privacy-safe training techniques can be used to reduce the overhead of encrypted inference for real-world datasets by leveraging transfer learning and differential privacy. Our experiments show that our method maintains competitive accuracy and achieves a significant speedup over previous methods. This work increases the viability of deep learning systems that use homomorphic encryption to protect user privacy.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Kim Laine
- Hao Chen
- M. Sadegh Riazi
- Sai Praneeth Karimireddy
- Lie He
- Martin Jaggi
- Blake Pelton
- Wei Dai
Citations in the corpus
- Secure, privacy-preserving and federated machine learning in medical imaging (2020, unknown): “…of the individual techniques already presented, efficient cryptographic and privacy primitives, neural network operations 88 based, for example, on functional encryption 89 , quantization 90 and optimization strategies 91 , and encrypted transfer learning approaches 92 must be further developed.”
- Privacy-Preserving Machine Learning With Fully Homomorphic Encryption for Deep Neural Network (2021, methodology): “HE-friendly network Some previous works re-design the machine learning model compatible with the HE scheme by replacing the standard activation functions with the simple non-linear polynomials [16, 17, 18, 19], called the HE-friendly network.”
- XONN: XNOR-based Oblivious Deep Neural Network Inference (2019, methodology): “Chou et al. propose to utilize the sparsity within the DL model to accelerate the inference [17].”
- HEAX: High-Performance Architecture for Computation on Homomorphically Encrypted Data in the Cloud (2019, background): “Privacy-preserving MLaaS has been studied rigorously in recent years and several protocol-level and algorithmic improvements have been proposed [36, 48, 63, 46, 19, 64, 62, 61, 42].”
- Secure Byzantine-Robust Machine Learning (2020, background): “[14, 33, 15, 22, 26, 33, 10, 16, 32] falls into a general category of 2-party computation (2PC).”