SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systems

By Edward Chou, Florian Tramèr, Giancarlo Pellegrino. 2018

352 citations recorded by the source corpus.

SentiNet is a novel detection framework for localized universal attacks on neural networks. These attacks restrict adversarial noise to contiguous portions of an image and are reusable with different images-constraints that prove useful for generating physically-realizable attacks. Unlike most other works on adversarial detection, SentiNet does not require training a model or preknowledge of an attack prior to detection. Our approach is appealing due to the large number of possible mechanisms and attack-vectors that an attack-specific defense would have to consider. By leveraging the neural network's susceptibility to attacks and by using techniques from model interpretability and object detection as detection mechanisms, SentiNet turns a weakness of a model into a strength. We demonstrate the effectiveness of SentiNet on three different attacks-i.e., data poisoning attacks, trojaned networks, and adversarial patches (including physically realizable attacks)-and show that our defense is able to achieve very competitive performance metrics for all three threats. Finally, we show that SentiNet is robust against strong adaptive adversaries, who build adversarial patches that specifically target the components of SentiNet's architecture.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Philip S. Yu
2. Lichao Sun
3. Bo Li
4. Lingjuan Lyu
5. Xingjun Ma
6. Tom Goldstein
7. Dawn Song
8. Qiang Yang

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Dacheng Tao
3. Tom Goldstein
4. Baoyuan Wu
5. Xiaogang Wang
6. Alan L. Yuille
7. Bo Li
8. Jiashi Feng

In Safety, Robustness & Interpretability

1. Bo Li
2. Xingjun Ma
3. Tom Goldstein
4. Aleksander Madry
5. Dawn Song
6. Dimitris Tsipras
7. Nicholas Carlini
8. Baoyuan Wu

Citations in the corpus

• Privacy and Robustness in Federated Learning: Attacks and Defenses (2020, methodology): “learning, for example, denoising (preprocessing) methods [33], [34], [35], [36], [37], backdoor sample/trigger detection methods [38], [39], [40], [41], [42], [43], robust data augmentations [44], and finetuning methods [44].”
• Invisible Backdoor Attack with Sample-Specific Triggers (2020, methodology): “For example, SentiNet [5] adopted the Grad-CAM [35] to extract critical regions from input towards each class and then located the trigger regions based on the boundary analysis.”
• Backdoor Learning: A Survey (2020, methodology): “ImageNet [88] 224× 224× 3 1,281,167 50,000 [44], [54], [68], [17], [18] [20], [71], [74], [84], [11] [10]”
• Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses (2020, methodology): “SentiNet [148] uses Grad-Cam [149], an input saliency mapping method, to pinpoint the features of the input that”
• NeuronInspect: Detecting Backdoors in Neural Networks via Output Explanations (2019, background): “SentiNet (Chou et al. 2018) is an attack agnostic framework detecting trojan backdoor attacks on DNN.”