Assessing Threat of Adversarial Examples on Deep Neural Networks

By Abigail Graese, Andras Rozsa, Terrance E. Boult. 2016

60 citations recorded by the source corpus.

Deep neural networks are facing a potential security threat from adversarial examples, inputs that look normal but cause an incorrect classification by the deep neural network. For example, the proposed threat could result in hand-written digits on a scanned check being incorrectly classified but looking normal when humans see them. This research assesses the extent to which adversarial examples pose a security threat, when one considers the normal image acquisition process. This process is mimicked by simulating the transformations that normally occur in of acquiring the image in a real world application, such as using a scanner to acquire digits for a check amount or using a camera in an autonomous car. These small transformations negate the effect of the carefully crafted perturbations of adversarial examples, resulting in a correct classification by the deep neural network. Thus just acquiring the image decreases the potential impact of the proposed security threat. We also show that the already widely used process of averaging over multiple crops neutralizes most adversarial examples. Normal preprocessing, such as text binarization, almost completely neutralizes adversarial examples. This is the first paper to show that for text driven classification, adversarial examples are an academic curiosity, not a security threat.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Hardware and Architecture

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Electrical and Electronic Engineering

In Hardware and Architecture

Citations in the corpus

Countering Adversarial Images using Input Transformations (2018, methodology): “We explore transformations based on image cropping and rescaling (Graese et al., 2016), bit-depth reduction (Xu et al.”
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey (2018, background): “[30], [48]) where this concern is down-played and adversarial examples are implicated to be ‘only a matter of curiosity’ with little practical concerns.”
Defense Against Adversarial Attacks Using High-Level Representation Guided Denoiser (2017, background): “[8] assess the defending performance of a set of preprocessing transformations on MNIST digits [17], including the perturbations introduced by image acquisition process, fusion of crops and binarization.”
Security and Privacy Issues in Deep Learning (2018, background): “Similarly, Guo et al. [2017] proposed a transformation approaches under the baseline of image cropping, rescaling [Graese et al. 2016], bit-depth reduction [Xu et al. 2017], JPEG compression [Kinga and Adam 2015], and total variance minimization [Rudin et al. 1992].”
Adversarial Image Perturbation for Privacy Protection A Game Theory Perspective (2017, background): “[7] argued that simple test time image processing, such as translation, Gaussian noise, blurring, and re-sizing, can equally neutralise the effect of AIPs, without having to retrain the convnet.”
Towards Robust Deep Neural Networks with BANG (2016, background): “[5] showed that transformations of the normal image acquisition process can also negate the effect of the carefully crafted adversarial perturbations.”
Attacks on Machine Learning: Lurking Danger for Accountability (2019, methodology): “Attack [4] Adversarial Examples [29, 31, 45, 52, 62, 70] Trojan Trigger [43] Multi-layer Perceptron Supervised Yes/No Poisoning Attack [46] No Equation-solving Attack [49] Ateniese et al.”