DarKnight: A Data Privacy Scheme for Training and Inference of Deep Neural Networks
By Hanieh Hashemi, Yongqin Wang, Murali Annavaram. 2020
27 citations recorded by the source corpus.
Protecting the privacy of input data is of growing importance as machine learning applications reach new application domains. Cloud companies are providing machine learning as a service to make it easier for data holders to create customized training and inference paradigms. In this paper, we provide a unified training and inference framework for large DNNs while protecting input privacy. Our approach called DarKnight relies on cooperative execution between GPUs and trusted execution environment (TEE) to train complex models. The cooperative execution allows DarKnight to exploit the computational power of GPUs to perform linear operations while exploiting TEEs to protect input privacy. In particular, DarKnight uses a novel encoding to linearly combine multiple inputs along with an additive stream cipher noise to obfuscate the inputs. The proposed encoding process allows DarKnight to efficiently decode the computed data even as the model parameters continuously evolve during the backward propagation of DNN training. DarKnight further simplifies the encoding process for inference where the model parameters are unchanged. Unlike prior approaches, DarKnight does not need to store model parameters within the TEE memory thereby getting around the TEE's limited secure memory limitations. By encoding and decoding multiple inputs during each iteration, DarKnight is well suited for the current generation batch training process. We implement DarKnight on an Intel SGX enclave augmented with a GPU to demonstrate our new training capabilities.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Murali Annavaram
- Qiang Yang
- Chaoyang He
- Yang Liu
- S. Avestimehr
- Praneeth Vepakomma
- Abhishek Singh
- Ramesh Raskar
Citations in the corpus
- FedML: A Research Library and Benchmark for Federated Machine Learning (2020, background): “To tackle the challenge of trustworthiness, existing research focuses on developing new defense techniques for adversarial attacks to make FL robust [15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 24, 27, 28], and proposing methods such as differential privacy (DP) and secure multiparty computation”
- Privacy in Deep Learning: A Survey (2020, methodology): “ partitioning to limit the attack surface against DNNs [153]. TEEs can provide integrity and confidentiality during execution. TEEs have been deployed in many forms, including Intel’s SGX-enabled CPUs [154, 155, 156, 157], Arm’s TrustZone [158]. This execution model, however, requires the users to se”