TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systems

By Wenbo Guo, Lun Wang, Xinyu Xing, Min Du, Dawn Song. 2019

245 citations recorded by the source corpus.

A trojan backdoor is a hidden pattern typically implanted in a deep neural network. It could be activated and thus forces that infected model behaving abnormally only when an input data sample with a particular trigger present is fed to that model. As such, given a deep neural network model and clean input samples, it is very challenging to inspect and determine the existence of a trojan backdoor. Recently, researchers design and develop several pioneering solutions to address this acute problem. They demonstrate the proposed techniques have a great potential in trojan detection. However, we show that none of these existing techniques completely address the problem. On the one hand, they mostly work under an unrealistic assumption (e.g. assuming availability of the contaminated training database). On the other hand, the proposed techniques cannot accurately detect the existence of trojan backdoors, nor restore high-fidelity trojan backdoor images, especially when the triggers pertaining to the trojan vary in size, shape and position. In this work, we propose TABOR, a new trojan detection technique. Conceptually, it formalizes a trojan detection task as a non-convex optimization problem, and the detection of a trojan backdoor as the task of resolving the optimization through an objective function. Different from the existing technique also modeling trojan detection as an optimization problem, TABOR designs a new objective function--under the guidance of explainable AI techniques as well as heuristics--that could guide optimization to identify a trojan backdoor in a more effective fashion. In addition, TABOR defines a new metric to measure the quality of a trojan backdoor identified. Using an anomaly detection method, we show the new metric could better facilitate TABOR to identify intentionally injected triggers in an infected model and filter out false alarms......

Subfields: Artificial Intelligence, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Electrical and Electronic Engineering

Citations in the corpus

STRIP: a defence against trojan attacks on deep neural networks (2019, methodology): “[34] propose the Neural Cleanse method to detect whether a DNN model has been trojaned or not prior to deployment, where its accuracy is further improved in [14].”
Invisible Backdoor Attacks on Deep Neural Networks Via Steganography and Regularization (2019, background): “However, the trigger generated in the Trojaning attack is so obvious that humans, N EURAL C LEANSE [3], and T ABOR [4] can detect it.”
Februus: Input Purification Defence Against Trojan Attacks on Deep Neural Network Systems (2019, result): “In contrast to existing investigations on Trojan detection [7,9,10,13,24] and cleaning [7, 13, 17, 24] a network, we seek to investigate the following questions:”
NeuronInspect: Detecting Backdoors in Neural Networks via Output Explanations (2019, methodology): “Previous trojan backdoor detection technique, such as Neural Cleanse and TABOR (Wang et al. ; Guo et al. 2019) restore the trigger pattern with the following optimization: argmin m L ,p L ( f ( Tr ( X, m L , p )) , y at ) .”
DAWN: Dynamic Adversarial Watermarking of Neural Networks (2019, methodology): “Most techniques like Neural Cleanse [45] and TABOR [18] can only detect backdoors for which the trigger is a static pattern added to original inputs (e.”
Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses (2020, methodology): “Moreover, generating additional models for trigger-agnosticmethods or reconstructingpossible triggers in trigger reconstruction methods [139], [141]”
Backdoor Learning: A Survey (2020, background): “ImageNet [88] 224× 224× 3 1,281,167 50,000 [44], [54], [68], [17], [18] [20], [71], [74], [84], [11] [10]”
Blind Backdoors in Deep Learning Models (2020, methodology): “Input perturbation NeuralCleanse [95], ABS [54], TABOR [30], STRIP [24], Neo [93], MESA [69], Titration analysis [21]”