When a Tree Falls: Using Diversity in Ensemble Classifiers to Identify Evasion in Malware Detectors

By Charles Smutz, Angelos Stavrou. 2016

100 citations recorded by the source corpus.

Machine learning classifiers are a vital component of modern malware and intrusion detection systems. However, past studies have shown that classifier based detection systems are susceptible to evasion attacks in practice. Improving the evasion resistance of learning based systems is an open problem. To address this, we introduce a novel method for identifying the observations on which an ensemble classifier performs poorly. During detection, when a sufficient number of votes from individual classifiers disagree, the ensemble classifier prediction is shown to be unreliable. The proposed method, ensemble classifier mutual agreement analysis, allows the detection of many forms of classifier evasion without additional external ground truth. We evaluate our approach using PDFrate, a PDF malware detector. Applying our method to data taken from a real network, we show that the vast majority of predictions can be made with high ensemble classifier agreement. However, most classifier evasion attempts, including nine targeted mimicry scenarios from two recent studies, are given an outcome of uncertain indicating that these observations cannot be given a reliable prediction by the classifier. To show the general applicability of our approach, we tested it against the Drebin Android malware detector where an uncertain prediction was correctly given to the majority of novel attacks. Our evaluation includes over 100,000 PDF documents and 100,000 Android applications. Furthermore, we show that our approach can be generalized to weaken the effectiveness of the Gradient Descent and Kernel Density Estimation attacks against Support Vector Machines. We discovered that feature bagging is the most important property for enabling ensemble classifier diversity based evasion detection.

Subfields: Artificial Intelligence, Computer Networks and Communications, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Signal Processing

Citations in the corpus

Adversarial Detection of Flash Malware: Limitations and Open Issues (2017, neutral): “[4, 5, 6, 7]).”
Dimensionality Reduction as a Defense against Evasion Attacks on Machine Learning Classifiers (2017, methodology): “An ensemble of classifiers was used by Smutz and Stavrou [43] to detect evasion attacks, by checking for disagreement between various classifiers.”
Digital Investigation of PDF Files: Unveiling Traces of Embedded Malware (2017, methodology): “Wepawet [6] Dynamic JSand JS-based Bayesian PJScan [7] Static Poppler JS-based SVM Hidost [3] Static Poppler Structural Random Forest Lux0R [8] Static PhoneyPDF JS-based Random Forest Slayer [1] Static PDFID Structural Random Forest Slayer NEO [2] Static PeePDF+Origami Structural Adaboost PDFRate [4”
Enhancing robustness of machine learning systems via data transformations (2017, methodology): “In such cases, it is likely our defense would have to be combined with other defenses such as adversarial training [16] and ensemble methods [45] for detection in order to create a ML system secure against evasion attacks.”
Towards Adversarial Malware Detection (2018, methodology): “Hence, we could only refer to what has been stated in the released papers [80, 82, 83, 89].”
PlatPal: Detecting Malicious Documents with Platform Diversity (2017, background): “[47] attempts to identify and prevent detection evasions by constructing diversified classifiers, ensembling them into a single system, and comparing their classification outputs with mutual agreement analysis.”