Randomized Prediction Games for Adversarial Machine Learning

By Samuel Rota Bulò, Battista Biggio, Ignazio Pillai, Marcello Pelillo, Fabio Roli. 2016

64 citations recorded by the source corpus.

In spam and malware detection, attackers exploit randomization to obfuscate malicious data and increase their chances of evading detection at test time, e.g., malware code is typically obfuscated using random strings or byte sequences to hide known exploits. Interestingly, randomization has also been proposed to improve security of learning algorithms against evasion attacks, as it results in hiding information about the classifier to the attacker. Recent work has proposed game-theoretical formulations to learn secure classifiers, by simulating different evasion attacks and modifying the classification function accordingly. However, both the classification function and the simulated data manipulations have been modeled in a deterministic manner, without accounting for any form of randomization. In this paper, we overcome this limitation by proposing a randomized prediction game, namely, a noncooperative game-theoretic formulation in which the classifier and the attacker make randomized strategy selections according to some probability distribution defined over the respective strategy set. We show that our approach allows one to improve the tradeoff between attack detection and false alarms with respect to the state-of-the-art secure classifiers, even against attacks that are different from those hypothesized during design, on application examples including handwritten digit recognition, spam, and malware detection.

Subfields: Artificial Intelligence, Information Systems, Safety, Robustness & Interpretability, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Safety, Robustness & Interpretability

In Signal Processing

Citations in the corpus

Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning (2017, background): “The latter approach has been ﬁrstly proposed in 2008 [43] as an e ﬀ ective way to hide information about the classiﬁcation function to the attacker, with recent follow-ups in [7, 45] to counter adversarial examples.”
Towards Adversarial Malware Detection (2018, background): “objective functions.10 In particular, while the attacker manipulates data within certain bounds to evade detection, the classifier is iteratively retrained to correctly detect the manipulated samples [16, 17, 38, 77]. We argue that game-theoretical models may be helpful to improve robustness of PDF ”
A Survey on Security Threats and Defensive Techniques of Machine Learning: A Data Driven View (2018, background): “[87] extended previous work and proposed a randomized prediction game by considering randomized strategy selections according to some probability distribution defined over the respective strategy set.”