Adversarial Reprogramming of Neural Networks

By Gamaleldin F. Elsayed, Ian J. Goodfellow, Jascha Narain Sohl-Dickstein. 2018

204 citations recorded by the source corpus.

Deep neural networks are susceptible to \emph{adversarial} attacks. In computer vision, well-crafted perturbations to images can cause neural networks to make mistakes such as confusing a cat with a computer. Previous adversarial attacks have been designed to degrade performance of models or cause machine learning models to produce specific outputs chosen ahead of time by the attacker. We introduce attacks that instead {\em reprogram} the target model to perform a task chosen by the attacker---without the attacker needing to specify or compute the desired output for each test-time input. This attack finds a single adversarial perturbation, that can be added to all test-time inputs to a machine learning model in order to cause the model to perform a task chosen by the adversary---even if the model was not trained to do this task. These perturbations can thus be considered a program for the new task. We demonstrate adversarial reprogramming on six ImageNet classification models, repurposing these models to perform a counting task, as well as classification tasks: classification of MNIST and CIFAR-10 examples presented as inputs to the ImageNet model.

Subfields: Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

Citations in the corpus

WARP: Word-level Adversarial ReProgramming (2021, methodology): “Apart from the different types of tasks and our characterization of the task as a form of Adversarial Reprogramming, the main difference between their approach and ours is that they use an additional parameterization trick to stabilize the training.”
Visual Prompt Tuning (2022, background): “this approach has been explored before in the adversarial attack literature [17].”
Adversarial Reprogramming of Text Classification Neural Networks (2018, methodology): “Adversarial Reprogramming (2019) studied whether it is possible to keep all the parameters of the neural network unchanged and simply learn an input transformation that can realign the outputs of the neural network for the new task.”
Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey (2019, methodology): “AP [31] is a recently proposed adversarial attack where an adversarial reprogramming function дθ is trained to re-purpose the attacked DNN to perform an alternate task (e.”
Exploring Visual Prompts for Adapting Large-Scale Models (2022, methodology): “Existing methods [12, 22, 5, 38], however, have focused on adversarial goals or demonstrated limited application to relatively small-scale datasets and models.”
Blind Backdoors in Deep Learning Models (2020, background): “Attacker’s access to model black-box [64], none* black-box [52] change data change model change code Attack modifies model no no yes yes yes Inference-time access required required required required optional Universal and small pattern no no yes yes yes Complex behavior limited [20] no no no yes Kno”
Advances in adversarial attacks and defenses in computer vision: A survey (2021, background): “[293] showed that adversarial perturbations can be used to reprogram a target model.”