ABS: Scanning Neural Networks for Back-doors by Artificial Brain Stimulation

By Yingqi Liu, Wen-Chuan Lee, Guanhong Tao, Shiqing Ma, Yousra Aafer, X. Zhang. 2019

558 citations recorded by the source corpus.

This paper presents a technique to scan neural network based AI models to determine if they are trojaned. Pre-trained AI models may contain back-doors that are injected through training or by transforming inner neuron weights. These trojaned models operate normally when regular inputs are provided, and mis-classify to a specific output label when the input is stamped with some special pattern called trojan trigger. We develop a novel technique that analyzes inner neuron behaviors by determining how output activations change when we introduce different levels of stimulation to a neuron. The neurons that substantially elevate the activation of a particular output label regardless of the provided input is considered potentially compromised. Trojan trigger is then reverse-engineered through an optimization procedure using the stimulation analysis results, to confirm that a neuron is truly compromised. We evaluate our system ABS on 177 trojaned models that are trojaned with various attack methods that target both the input space and the feature space, and have various trojan trigger sizes and shapes, together with 144 benign models that are trained with different data and initial weight values. These models belong to 7 different model structures and 6 different datasets, including some complex ones such as ImageNet, VGG-Face and ResNet110. Our results show that ABS is highly effective, can achieve over 90% detection rate for most cases (and many 100%), when only one input sample is provided for each output label. It substantially out-performs the state-of-the-art technique Neural Cleanse that requires a lot of input samples and small trojan triggers to achieve good performance.

Subfields: Artificial Intelligence, Cellular and Molecular Neuroscience, Hardware and Architecture

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Hardware and Architecture

Citations in the corpus

Invisible Backdoor Attacks on Deep Neural Networks Via Steganography and Regularization (2019, methodology): “As without a clean reference model (a reasonable assumption because the defender only has the provided DNN model in hand [4, 46]), the anomaly detection of the Neural Cleanse (MAD) will fail.”
Poisoning Web-Scale Training Datasets is Practical (2023, unknown): “…an exceptionally vast literature [1], [3], [7], [10], [11], [17], [18], [22], [31], [54], [62], [86], [99], [106], [113], [26], [30], [37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29], [57], [65],…”
Demon in the Variant: Statistical Analysis of DNNs for Robust Backdoor Contamination Detection (2019, background): “A new solution recently proposed is ABS [23], which detects compromised backdoor neurons from a large difference in their activation with or without a Trojaned image.”
Blind Backdoors in Deep Learning Models (2020, methodology): “Input perturbation NeuralCleanse [95], ABS [54], TABOR [30], STRIP [24], Neo [93], MESA [69], Titration analysis [21]”
BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvements (2020, background): “Existing defenses against backdoor attacks such as ABS [18], Neural Cleanse [41] and STRIP [9] are primarily designed for the image domain.”
Input-Aware Dynamic Backdoor Attack (2020, methodology): “ABS (6), instead, scanned the neurons and generated trigger candidates via reverse engineering technique.”
Privacy-Enhanced Federated Learning Against Poisoning Adversaries (2021, background): “Recently, some researchers have proposed to disconnect the end-to-end mapping from the speciﬁc feature space to the target label space from the neuron level [38], [39].”
Towards Probabilistic Verification of Machine Unlearning (2020, methodology): “For instance, in our backdoor, when using EMNIST dataset, each image is n = 784 = 28 × 28 , we have set w = 4 pixels and d = 2 (i.e., if two backdoors share 3 of the 4 pixels, they interfere with each others classiﬁcation).”