STRIP: a defence against trojan attacks on deep neural networks

By Yansong Gao, Chang Xu, Derui Wang, Shiping Chen, Damith Chinthana Ranasinghe, Surya Nepal. 2019

1,077 citations recorded by the source corpus.

A recent trojan attack on deep neural network (DNN) models is one insidious variant of data poisoning attacks. Trojan attacks exploit an effective backdoor created in a DNN model by leveraging the difficulty in interpretability of the learned model to misclassify any inputs signed with the attacker's chosen trojan trigger. Since the trojan trigger is a secret guarded and exploited by the attacker, detecting such trojan inputs is a challenge, especially at run-time when models are in active operation. This work builds STRong Intentional Perturbation (STRIP) based run-time trojan attack detection system and focuses on vision system. We intentionally perturb the incoming input, for instance by superimposing various image patterns, and observe the randomness of predicted classes for perturbed inputs from a given deployed model---malicious or benign. A low entropy in predicted classes violates the input-dependence property of a benign model and implies the presence of a malicious input---a characteristic of a trojaned input. The high efficacy of our method is validated through case studies on three popular and contrasting datasets: MNIST, CIFAR10 and GTSRB. We achieve an overall false acceptance rate (FAR) of less than 1%, given a preset false rejection rate (FRR) of 1%, for different types of triggers. Using CIFAR10 and GTSRB, we have empirically achieved result of 0% for both FRR and FAR. We have also evaluated STRIP robustness against a number of trojan attack variants and adaptive attacks.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

In Signal Processing

Citations in the corpus

You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion (2020, background): “Many defenses have been proposed in the literature [12, 15, 18, 22, 27, 29, 35, 36, 46, 58, 59, 62, 65, 66].”
How To Backdoor Federated Learning (2018, methodology): “Similarly, STRIP [19] and DeepInspect [9] only target pixel-pattern backdoors.”
BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvements (2020, background): “Existing defenses against backdoor attacks such as ABS [18], Neural Cleanse [41] and STRIP [9] are primarily designed for the image domain.”
Demon in the Variant: Statistical Analysis of DNNs for Robust Backdoor Contamination Detection (2019, methodology): “Other attempts analyze a target model’s behavior towards a synthesized image created by blending those with different labels [10], or images with and without triggers [9], to determine the presence of a backdoor.”
Blind Backdoors in Deep Learning Models (2020, methodology): “Input perturbation NeuralCleanse [95], ABS [54], TABOR [30], STRIP [24], Neo [93], MESA [69], Titration analysis [21]”
Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks (2021, methodology): “Detection-based methods aim at identifying the existence of backdoor adversaries in the underlying model (Wang et al., 2019a; Kolouri et al., 2020) or ﬁltering the suspicious samples from input data for re-training (Tran et al., 2018; Gao et al., 2019; Chen et al., 2019b).”
Invisible Backdoor Attacks on Deep Neural Networks Via Steganography and Regularization (2019, neutral): “In STRIP [49], the authors rely on the strength characteristic of the backdoor attack, which is “image-agnostic”.”
Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses (2020, background): “STRIP [147] detects whether a given input contains a backdoor trigger by mixing it with other benign inputs and analyzing the model prediction.”
Adversarial Neuron Pruning Purifies Backdoored Deep Models (2021, methodology): “Regarding poisoned data as outliers, we can detect and filter them out using robust statistics in the input space [38, 18, 8, 9] or the feature space [17, 41, 28, 10].”
Input-Aware Dynamic Backdoor Attack (2020, methodology): “STRIP (7) exploited the persistent outcome of the backdoor image under perturbations for detection.”
TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systems (2019, background): “Gao et al. introduce STRIP [15], a detection system that examines the prediction variation and thus pinpoints the input data samples with the presence of the malicious trigger.”
Februus: Input Purification Defence Against Trojan Attacks on Deep Neural Network Systems (2019, background): “Identical to the approach of recent papers [9, 10, 24], we focus on input-agnostic attacks while the trigger will misclassify any inputs to a targeted class regardless input sources (illustrated in Figure 2).”
Detecting AI Trojans Using Meta Neural Analysis (2019, methodology): “This definition of black-box model is widely used in existing work [45], [14], [52], [21].”
Invisible Backdoor Attack with Sample-Specific Triggers (2020, methodology): “Resistance to STRIP.”
Dynamic Backdoor Attacks Against Machine Learning Models (2020, methodology): “[10] propose STRIP, a backdoor defense method based on manipulating the input, to find out if it is backdoored or not.”
Model Agnostic Defence Against Backdoor Attacks in Machine Learning (2019, background): “Another defence [5] leverages entropy to detect backdoor attacks.”