Adversarial Perturbations Against Deep Neural Networks for Malware Classification

By Kathrin Grosse, Nicolas Papernot, Praveen Manoharan, Michael Backes, Patrick Drew McDaniel. 2016

451 citations recorded by the source corpus.

Deep neural networks, like many other machine learning models, have recently been shown to lack robustness against adversarially crafted inputs. These inputs are derived from regular inputs by minor yet carefully selected perturbations that deceive machine learning models into desired misclassifications. Existing work in this emerging field was largely specific to the domain of image classification, since the high-entropy of images can be conveniently manipulated without changing the images' overall visual appearance. Yet, it remains unclear how such attacks translate to more security-sensitive applications such as malware detection - which may pose significant challenges in sample generation and arguably grave consequences for failure. In this paper, we show how to construct highly-effective adversarial sample crafting attacks for neural networks used as malware classifiers. The application domain of malware classification introduces additional constraints in the adversarial sample crafting problem when compared to the computer vision domain: (i) continuous, differentiable input domains are replaced by discrete, often binary inputs; and (ii) the loose condition of leaving visual appearance unchanged is replaced by requiring equivalent functional behavior. We demonstrate the feasibility of these attacks on many different instances of malware classifiers that we trained using the DREBIN Android malware data set. We furthermore evaluate to which extent potential defensive mechanisms against adversarial crafting can be leveraged to the setting of malware classification. While feature reduction did not prove to have a positive impact, distillation and re-training on adversarially crafted samples show promising results.

Subfields: Artificial Intelligence, Computer Networks and Communications, Safety, Robustness & Interpretability, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Safety, Robustness & Interpretability

In Signal Processing

Citations in the corpus

Towards the Science of Security and Privacy in Machine Learning (2016, background): “White-Box Full No [51], [52], [53] [30], [26], [54] [55] Through pipeline only No [56], [57], [37] [37]”
ZOO: Zeroth Order Optimization Based Black-box Attacks to Deep Neural Networks without Training Substitute Models (2017, methodology): “In addition to image classification, JSMA has been applied to other machine learning tasks such as malware classification [14], and other DNN”
When Does Machine Learning FAIL? Generalized Transferability for Evasion and Poisoning Attacks (2018, methodology): “We generate these applications for the scenario where only the subset of features extracted from the AndroidManifest are modifiable by the attacker, similar to prior work [19].”
On the (Statistical) Detection of Adversarial Examples (2017, methodology): “An example is classified by finding a path of appropriate conditions from the root to one of the leaves.”
AI2: Safety and Robustness Certification of Neural Networks with Abstract Interpretation (2018, background): “, [3], [12], [14], [15], [17], [18], [29], [30], [32], [38], [41]).”
Spatially Transformed Adversarial Examples (2018, background): “…can be used to subvert malware detection, fraud detection, or even potentially mislead autonomous navigation systems (Papernot et al., 2016b; Evtimov et al., 2017; Grosse et al., 2016; Li & Vorobeychik, 2014; 2015) and therefore pose security risks when applied to security-related applications.”
MagNet: A Two-Pronged Defense against Adversarial Examples (2017, background): “Recent work has demonstrated the feasibility of attacking such systems with carefully crafted input for real-world systems [2, 28, 8].”
DeepXplore: Automated Whitebox Testing of Deep Learning Systems (2017, background): “demonstrated the existence of some corner cases where DNNbased image classifiers (with state-of-the-art performance on randomly picked testing sets) still incorrectly classify synthetic images generated by adding humanly imperceptible perturbations to a test image [26, 29, 52, 63, 79, 85].”
Early Methods for Detecting Adversarial Images (2016, methodology): “Other fooling data could evade malware detectors or spam ﬁlters that use a deep learning backend (Grosse et al., 2016).”
Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning (2018, background): “Note that this is not the case in other application domains, such as adversarial examples for malware detection [111], [6], where an algorithmic oracle is often available—such as a virtual machine running the executable as shown in Xu et al.”
Adversarially Robust Generalization Requires More Data (2018, background): “More recently, researchers have demonstrated the phenomenon under the name of adversarial examples in image classification [21, 51], question answering [28], voice recognition [12, 13, 49, 62], and other domains (for instance, see [2, 4, 14, 22, 25, 26, 32, 60]).”
One Pixel Attack for Fooling Deep Neural Networks (2017, background): “perturbation can be also possible in other domains, such as speech recognition [33], natural language processing [34] and malware classification [35].”
The Space of Transferable Adversarial Examples (2017, background): “Adversarial examples can be used to craft human-recognizable images that are misclassified by computer vision models [23, 8, 12, 16], software containing malware but classified as benign [13, 26, 9, 10], and game environments that force reinforcement learning agents to misbehave [11, 4, 15].”
Motivating the Rules of the Game for Adversarial Example Research (2018, background): “which specifically consider defense strategies for increasing neural network robustness to restricted l0 changes to a feature vector constructed from a malware binary [91, 101].”
SoK: Security and Privacy in Machine Learning (2018, methodology): “For instance, when crafting malware that evades detection by a ML system, it is easier to introduce perturbations r that only modify a limited subset of the features, rather than making small changes to all features [49].”
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey (2018, methodology): “[61] showed that creating effective adversarial examples is still possible for maleware classification.”
Adversarial Risk and the Dangers of Evaluating Against Weak Attacks (2018, background): “…examples are the most straightforward to study in the visual domain, they have also been demonstrated in other domains such as reinforcement learning (Huang et al., 2017), robotics (Pinto et al., 2017), malware detection (Grosse et al., 2016), and natural language processing (Jia and Liang, 2017).”