Automatically Evading Classifiers: A Case Study on PDF Malware Classifiers

By Weilin Xu, Yanjun Qi, David E. Evans. 2016

249 citations recorded by the source corpus.

Machine learning is widely used to develop classifiers for security tasks. However, the robustness of these methods against motivated adversaries is uncertain. In this work, we propose a generic method to evaluate the robustness of classifiers under attack. The key idea is to stochastically manipulate a malicious sample to find a variant that preserves the malicious behavior but is classified as benign by the classifier. We present a general approach to search for evasive variants and report on results from experiments using our techniques against two PDF malware classifiers, PDFrate and Hidost. Our method is able to automatically find evasive variants for both classifiers for all of the 500 malicious seeds in our study. Our results suggest a general method for evaluating classifiers used in security applications, and raise serious doubts about the effectiveness of classifiers based on superficial features in the presence of adversaries.

Subfields: Computer Networks and Communications, Information Systems, Safety, Robustness & Interpretability, Privacy & Security, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Information Systems

In Safety, Robustness & Interpretability

In Privacy & Security

In Signal Processing

Citations in the corpus

Attacking Graph-based Classification via Manipulating the Graph Structure (2019, background): “ing studies on adversarial machine learning mainly focused on machine learning for non-graph data. In particular, studies have demonstrated that machine learning is vulnerable to adversarial examples [6, 12, 28, 31, 35, 42, 45, 48, 53, 55, 58, 71], poisoning attacks [7, 19, 32, 39, 70, 73], privacy ”
Stealing Hyperparameters in Machine Learning (2018, background): “[56] proposed a general evasion attack to search for a malware variant that preserves the malicious behavior of the malware but is classified as benign by the classifier (e.”
Evasion and Hardening of Tree Ensemble Classifiers (2015, methodology): “In Xu et al. (Xu et al., 2016), the authors present a genetic algorithm for ﬁnding malicious PDF instances which evade detection.”
SoK: Security and Privacy in Machine Learning (2018, background): “Black-Box Yes No [51] [30], [47] [52], [29] [53]”
Feature Conservation in Adversarial Classifier Evasion: A Case Study (2017, methodology): “EvadeML was used to evade SL2013 in [37].”
A Survey on Security Threats and Defensive Techniques of Machine Learning: A Data Driven View (2018, background): “Similar to the above attack, the impersonate attack prefers to imitate data samples from victims, particularly in application scenarios of image recognition [59], malware detection [60], [61], intrusion detection [62] based on machine learning.”
NEZHA: Efficient Domain-Independent Differential Testing (2017, background): “Recent works have applied differential testing to search for inputs that can evade machine learning classifiers for malware detection [46], [64].”
Why Do Adversarial Attacks Transfer? Explaining Transferability of Evasion and Poisoning Attacks (2018, background): “the other hand, evasion attacks involve small manipulations of testing data points that results in misprediction at testing time on those points [3, 8, 10, 14, 32, 38, 42, 45, 49].”