Evading Classifiers by Morphing in the Dark

By Hung Dang, Yue Huang, Ee-Chien Chang. 2017

126 citations recorded by the source corpus.

Learning-based systems have been shown to be vulnerable to evasion through adversarial data manipulation. These attacks have been studied under assumptions that the adversary has certain knowledge of either the target model internals, its training dataset or at least classification scores it assigns to input samples. In this paper, we investigate a much more constrained and realistic attack scenario wherein the target classifier is minimally exposed to the adversary, revealing only its final classification decision (e.g., reject or accept an input sample). Moreover, the adversary can only manipulate malicious samples using a blackbox morpher. That is, the adversary has to evade the targeted classifier by morphing malicious samples "in the dark". We present a scoring mechanism that can assign a real-value score which reflects evasion progress to each sample based on the limited information available. Leveraging on such scoring mechanism, we propose an evasion method -- EvadeHC? and evaluate it against two PDF malware detectors, namely PDFRate and Hidost. The experimental evaluation demonstrates that the proposed evasion attacks are effective, attaining 100% evasion rate on the evaluation dataset. Interestingly, EvadeHC outperforms the known classifier evasion techniques that operate based on classification scores output by the classifiers. Although our evaluations are conducted on PDF malware classifiers, the proposed approaches are domain agnostic and are of wider application to other learning-based systems.

Subfields: Artificial Intelligence, Computer Networks and Communications, Safety, Robustness & Interpretability, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Safety, Robustness & Interpretability

In Signal Processing

Citations in the corpus

Adversarial Detection of Flash Malware: Limitations and Open Issues (2017, methodology): “We then evaluate FlashBuster robustness by simulating evasion attacks that leverage the knowledge that the attacker may possess about the targeted learning system [9, 8, 27, 19, 28, 29, 30], against an increasing number of modifications to the input samples.”
One Pixel Attack for Fooling Deep Neural Networks (2017, background): “Several black-box attacks that require no knowledge about the target systems such as gradients have also been proposed [?][21][13].”
Trojaning Attack on Neural Networks (2018, background): “[24] relax the attack scenario on classification evasion attack.”
Intriguing Properties of Adversarial ML Attacks in the Problem Space (2019, background): “Existing work investigated problem-space attacks on text [3, 43], malicious PDFs [12, 22, 41, 45, 46, 74], Android malware [23, 75], Windows malware [38, 60], NIDS [6, 7, 20, 28], ICS [76], source code attribution [58], malicious Javascript [27], and eyeglass frames [62].”
Attacks on Machine Learning: Lurking Danger for Accountability (2019, background): “Another group of attacks that tamper with data fed into the ML model, and thus are applicable on a wide range of different ML algorithms, are adversarial examples [5, 6, 17, 34, 51, 59], evasion attacks [23, 78], and feature deletion attacks.”
Understanding and Controlling User Linkability in Decentralized Learning (2018, background): “For instance, these models can be analyzed to recover private information about the users or the model itself [14, 21, 50, 66].”
Adversarial Neural Network Inversion via Auxiliary Knowledge Alignment (2019, background): “Various works have suggested that ML models are likely vulnerable in adversarial settings [50], [51], [40].”
Towards Adversarial Malware Detection (2018, background): “Note that using surrogate classifiers is not strictly necessary here [21, 27, 92, 102]; however, it is possible to learn a surrogate classifier (potentially on a different feature representation) and check whether the crafted attack samples transfer to the target classifier.”
Understanding the One Pixel Attack: Propagation Maps and Locality Analysis (2019, unknown): “To cite some (Narodytska & Kasiviswanathan, 2017),(Papernot et al., 2017),(Dang et al., 2017).”
Gradient-Leaks: Understanding and Controlling Deanonymization in Federated Learning (2018, background): “arge amounts of data, there is a push [2, 55, 59, 66, 68] in addressing the modelprivacy. For instance, these models can be analyzed to recover private information about the users or the model itself [14, 21, 50, 66]. Our work is closest to the model privacy threats where the adversary attempts to l”
Neural Network Inversion in Adversarial Setting via Background Knowledge Alignment (2019, background): “Various works have suggested that ML models are likely vulnerable in adversarial settings [14, 16, 25].”
Exploring Backdoor Poisoning Attacks Against Malware Classifiers (2020, methodology): “[22] introduced EvadeHC, performing a black-box attack by morphing the malicious file using a hill-climbing approach.”
Why Do Adversarial Attacks Transfer? Explaining Transferability of Evasion and Poisoning Attacks (2018, background): “the other hand, evasion attacks involve small manipulations of testing data points that results in misprediction at testing time on those points [3, 8, 10, 14, 32, 38, 42, 45, 49].”
Better the Devil you Know: An Analysis of Evasion Attacks using Out-of-Distribution Adversarial Examples (2019, background): “Black-box with query access: This is a more restrictive threat model, where the adversary is only assumed to have access to the output probability distribution д(·) for any input x ∈ X [12, 25, 31, 60, 93].”