Cache Telepathy: Leveraging Shared Resource Attacks to Learn DNN Architectures

By Mengjia Yan, Christopher W. Fletcher, Josep Torrellas. 2018

289 citations recorded by the source corpus.

Deep Neural Networks (DNNs) are fast becoming ubiquitous for their ability to attain good accuracy in various machine learning tasks. A DNN's architecture (i.e., its hyper-parameters) broadly determines the DNN's accuracy and performance, and is often confidential. Attacking a DNN in the cloud to obtain its architecture can potentially provide major commercial value. Further, attaining a DNN's architecture facilitates other, existing DNN attacks. This paper presents Cache Telepathy: a fast and accurate mechanism to steal a DNN's architecture using the cache side channel. Our attack is based on the insight that DNN inference relies heavily on tiled GEMM (Generalized Matrix Multiply), and that DNN architecture parameters determine the number of GEMM calls and the dimensions of the matrices used in the GEMM functions. Such information can be leaked through the cache side channel. This paper uses Prime+Probe and Flush+Reload to attack VGG and ResNet DNNs running OpenBLAS and Intel MKL libraries. Our attack is effective in helping obtain the architectures by very substantially reducing the search space of target DNN architectures. For example, for VGG using OpenBLAS, it reduces the search space from more than $10^{35}$ architectures to just 16.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Electrical and Electronic Engineering

Citations in the corpus

Privacy in Deep Learning: A Survey (2020, background): “Inference Parameter Inference Property Inference Access to Model Access to Model Output Membership Inference [13] # # # # Blackbox Logits Measuring Membership Privacy[35] # # # # Blackbox Logits ML-Leaks[36] # # # # Blackbox Logits The Natural Auditor [37] # # # # Blackbox Label LOGAN [38] # # # # B”
When Relaxations Go Bad: "Differentially-Private" Machine Learning (2019, unknown): “Hyperparameter stealing attacks try to recover the underlying hyperparameters used during the model training, such as regularization coe ﬃ cient [69] or model architecture [74].”
A framework for the extraction of Deep Neural Networks by leveraging public data (2019, methodology): “[20] use a similar insight that the forward pass of DNNs rely on GeMM (generalized matrix multiply) library operations.”
Survey of Attacks and Defenses on Edge-Deployed Neural Networks (2019, methodology): “Cache Telepathy [38] suggests less aggressive compiler optimizations, cache partitioning [67] or disallowing resource sharing as defenses against cache-based SC.”
Evaluating Differentially Private Machine Learning in Practice (2019, background): “Hyperparameter stealing attacks try to recover the underlying hyperparameters used during the model training, such as regularization coefficient [68] or model architecture [73].”