Explaining Black-box Android Malware Detection

By Marco Melis, Davide Maiorca, Battista Biggio, Giorgio Giacinto, Fabio Roli. 2018

49 citations recorded by the source corpus.

Machine-learning models have been recently used for detecting malicious Android applications, reporting impressive performances on benchmark datasets, even when trained only on features statically extracted from the application, such as system calls and permissions. However, recent findings have highlighted the fragility of such in-vitro evaluations with benchmark datasets, showing that very few changes to the content of Android malware may suffice to evade detection. How can we thus trust that a malware detector performing well on benchmark data will continue to do so when deployed in an operating environment? To mitigate this issue, the most popular Android malware detectors use linear, explainable machine-learning models to easily identify the most influential features contributing to each decision. In this work, we generalize this approach to any black-box machine-learning model, by leveraging a gradient-based approach to identify the most influential local features. This enables using nonlinear models to potentially increase accuracy without sacrificing interpretability of decisions. Our approach also highlights the global characteristics learned by the model to discriminate between benign and malware applications. Finally, as shown by our empirical analysis on a popular Android malware detection task, it also helps identifying potential vulnerabilities of linear and nonlinear models against adversarial manipulations.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Adversarial Detection of Flash Malware: Limitations and Open Issues (2017, background): “These assessments have been recently expanded to systems for Android malware detection [21, 22, 23, 47, 18] and to binary executables (with the adoption of deep learning) [48, 49].”
Explainable Artificial Intelligence Applications in Cyber Security: State-of-the-Art in Research (2022, methodology): “[152] implemented a gradient-based approach to identify the most influential features contributing to each decision.”
Towards Adversarial Malware Detection (2018, methodology): “Developing specific learning approaches that distribute feature importance more evenly can significantly improve robustness to such adversarial evasion attempts [11, 28, 50, 67].”
TESSERACT: Eliminating Experimental Bias in Malware Classification across Space and Time (2018, background): “Recent results have also shown that linear SVM tends to overemphasize a few important features [35]—which are the few most effective on the training data, but may become obsolete over time.”
Intriguing Properties of Adversarial ML Attacks in the Problem Space (2019, result): “This performance retention follows from recent results that shows SVM tends to overemphasize a subset of features [48].”
On the effectiveness of system API-related information for Android ransomware detection (2018, background): “edu/projects/revealdroid/ can make the system more robust against them (the attacker can manipulate less information to evade the system) [21].”