Towards Reverse-Engineering Black-Box Neural Networks

By Seong Joon Oh, Maximilian Augustin, Mario Fritz, Bernt Schiele. 2017

405 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks (2018, background): “Related work [39] also makes a similar argument that it can be useful to steal hyperparameters in order to mount more powerful attacks on models.”
Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, neutral): “An approach that has served to make DL models more robust against intellectual property exposure based on a sequence of non accessible queries is in [357] .”
Exploiting Unintended Feature Leakage in Collaborative Learning (2018, background): “Training data that is explicitly incorporated or otherwise memorized in the model can also be leaked by model stealing attacks [41, 57, 61].”
Property Inference Attacks on Fully Connected Neural Networks using Permutation Invariant Representations (2018, background): “[31] show that they can successfully reverse-engineer internal information such as the architecture and hyperparameters of blackbox neural network models.”
Disentangling Adversarial Robustness and Generalization (2018, methodology): “using model stealing [87, 70, 101, 103, 69, 43]) and have, thus, become standard for evaluation.”
A framework for the extraction of Deep Neural Networks by leveraging public data (2019, methodology): “However, as peak performance is achieved when the adversary is aware of the architecture of the secret model, and since it is possible to detect these hyperparameters and architectural choices by a related line of work (model reverse-engineering [17, 18, 19, 20, 21, 22]), we report our main results ”
Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learning (2019, background): “In addition to model parameters, several works concentrate on stealing ML models’ hyperparameters [33, 47].”
A Survey of Privacy Attacks in Machine Learning (2020, methodology): “Such an active attacker reached an attack accuracy of 92.1%.”
MemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial Examples (2019, background): “Another line of works studied hyperparameter stealing attacks [44, 65], which aim to steal the hyperparameters such as the neural network architecture and the hyperparameter that balances between the loss function and the regularization term.”
Privacy-preserving Machine Learning through Data Obfuscation (2018, background): “[25] built metamodels to extract more model details such as neural network architecture, which is assumed public in [35].”
High-Fidelity Extraction of Neural Network Models (2019, background): “It is also possible to query a model to steal hyperparam-eters [51] or architectural information [14].”
Attributing Fake Images to GANs: Learning and Analyzing GAN Fingerprints (2018, methodology): “We apply five types of attacks that perturb testing images [48]: noise, blur, cropping, JPEG compression, relighting, and random combination of them.”
Inference Attacks Against Collaborative Learning (2018, neutral): “Other attacks include model stealing [36, 49, 52].”
Improving Black-box Adversarial Attacks with a Transfer-based Prior (2019, background): “A meta-model [25] can reverse-engineer the black-box model and predict its attributes (such as architecture, optimization procedure, and training data) through a sequence of queries.”
Adversarial frontier stitching for remote neural network watermarking (2017, neutral): “Other attacks focus on the stealing of model hyperpa-rameters, from APIs [37,23]; [23] aims at infering inner hyperparameters ( e.g. , number of layers, non-linear activation type) of a remote neural network model by analysing its response patterns to certain inputs.”
MLCapsule: Guarded Offline Deployment of Machine Learning as a Service (2018, methodology): “A series of recent papers have shown different attacks on MLaaS that can lead to reverse engineering [35, 22] and training data leakage [12, 11, 29, 38, 28].”
Model inversion attacks against collaborative inference (2019, background): “The adversary attempts to steal the model parameters [44], hyperparameters [46] or structures [23, 33], via prediction APIs, memory access side channels, etc.”
Gradient-Leaks: Understanding and Controlling Deanonymization in Federated Learning (2018, background): “arge amounts of data, there is a push [2, 55, 59, 66, 68] in addressing the modelprivacy. For instance, these models can be analyzed to recover private information about the users or the model itself [14, 21, 50, 66]. Our work is closest to the model privacy threats where the adversary attempts to l”
Dynamic Backdoor Attacks Against Machine Learning Models (2020, background): “[37], model stealing attacks [44], [29], [46], data poisoning attacks [5], [17], [42], Trojan attacks [22], and backdoor attacks [49], [12].”