Model Agnostic Defence Against Backdoor Attacks in Machine Learning

By Sakshi Udeshi, Shanshan Peng, Gerald Woo, Lionell Loh, L. Rawshan, Sudipta Chattopadhyay. 2019

125 citations recorded by the source corpus.

Machine learning (ML) has automated a multitude of our day-to-day decision-making domains, such as education, employment, and driving automation. The continued success of ML largely depends on our ability to trust the model we are using. Recently, a new class of attacks called backdoor attacks have been developed. These attacks undermine the user’s trust in ML models. In this article, we present Neo, a model agnostic framework to detect and mitigate such backdoor attacks in image classification ML models. For a given image classification model, our approach analyzes the inputs it receives and determines if the model is backdoored. In addition to this feature, we also mitigate these attacks by determining the correct predictions of the poisoned images. We have implemented Neo and evaluated it against three state-of-the-art poisoned models. In our evaluation, we show that Neo can detect $\approx$88% of the poisoned inputs on average and it is as fast as 4.4 ms per input image. We also compare our Neo approach with the state-of-the-art defence methodologies proposed for backdoor attacks. Our evaluation reveals that despite being a blackbox approach, Neo is more effective in thwarting backdoor attacks than the existing techniques. Finally, we also reconstruct the exact poisoned input for the user to effectively test their systems.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

In Signal Processing

Citations in the corpus

You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion (2020, background): “Many defenses have been proposed in the literature [12, 15, 18, 22, 27, 29, 35, 36, 46, 58, 59, 62, 65, 66].”
Backdoor Learning: A Survey (2020, methodology): “6 million images of 2,622 people [23], [48], [52], [49], [43] [71] VGGFace2 [94] — 3.”
Input-Aware Dynamic Backdoor Attack (2020, background): “In contrast, Neo (8) searched for the candidate trigger patches where region blocking led to prediction change.”
Privacy and Robustness in Federated Learning: Attacks and Defenses (2020, methodology): “learning, for example, denoising (preprocessing) methods [33], [34], [35], [36], [37], backdoor sample/trigger detection methods [38], [39], [40], [41], [42], [43], robust data augmentations [44], and finetuning methods [44].”
Blind Backdoors in Deep Learning Models (2020, methodology): “Input perturbation NeuralCleanse [95], ABS [54], TABOR [30], STRIP [24], Neo [93], MESA [69], Titration analysis [21]”