Is data clustering in adversarial settings secure?

By Battista Biggio, Ignazio Pillai, Samuel Rota Bulò, Davide Ariu, Marcello Pelillo, Fabio Roli. 2013

135 citations recorded by the source corpus.

Clustering algorithms have been increasingly adopted in security applications to spot dangerous or illicit activities. However, they have not been originally devised to deal with deliberate attack attempts that may aim to subvert the clustering process itself. Whether clustering can be safely adopted in such settings remains thus questionable. In this work we propose a general framework that allows one to identify potential attacks against clustering algorithms, and to evaluate their impact, by making specific assumptions on the adversary's goal, knowledge of the attacked system, and capabilities of manipulating the input data. We show that an attacker may significantly poison the whole clustering process by adding a relatively small percentage of attack samples to the input data, and that some attack samples may be obfuscated to be hidden within some existing clusters. We present a case study on single-linkage hierarchical clustering, and report experiments on clustering of malware samples and handwritten digits.

Subfields: Artificial Intelligence, Computer Networks and Communications, Privacy & Security, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Privacy & Security

In Signal Processing

Citations in the corpus

Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, neutral): “There are even available solutions for unsupervised ML, like clustering algorithms [363] .”
Certified Defenses for Data Poisoning Attacks (2017, background): “, 2015a), clustering (Biggio et al., 2013; 2014c), PCA (Rubinstein et al.”
Towards Poisoning of Deep Learning Algorithms with Back-gradient Optimization (2017, background): “Here we do not follow their naming convention, as it can cause confusion with the interpretation of targeted and indiscriminate attacks introduced in previous work [1, 2, 4, 7, 9, 10, 16, 39].”
Poisoning behavioral malware clustering (2014, methodology): “…goal, knowledge and capabilities, we can ﬁnally deﬁne the optimal strategy for attacking a clustering algorithm as: In this formulation, as in [4, 9], the adversary’s knowledge is characterized by a parameter vector θ , whose elements embed information about the input data D , the clustering…”
Adversarial Feature Selection Against Evasion Attacks (2016, background): “Previous work on evasion attacks has mainly investigated the vulnerability of supervised [7], [11], [20] and unsupervised learning techniques [21], [22] in different applications, including spam filtering [5], [23], [24], intrusion detection [25], and biometric recognition [1], [2].”
Support vector machines under adversarial label contamination (2015, background): “Recently, poisoning and evasion attacks against clustering algorithms have also been formalized to show that malware clustering approaches can be significantly vulnerable to well-crafted attacks [18, 19].”
Poisoning and Backdooring Contrastive Learning (2021, background): “Poisoning attacks exist for many tasks, including supervised (Biggio et al., 2012; Turner et al., 2019; Koh & Liang, 2017), unsupervised (Kloft & Laskov, 2010; 2012; Biggio et al., 2013), and semi-supervised (Liu et al., 2020; Carlini, 2021) learning.”
Pattern Recognition Systems under Attack (2013, methodology): “We have recently demonstrated that an attacker may significantly subvert the whole clustering process by adding a relatively small percentage of attack samples to the input data [8].”
Attack Strength vs. Detectability Dilemma in Adversarial Machine Learning (2018, methodology): “Poisoning attacks have also been developed against clustering algorithms [20], and have been shown to be effective against feature selection algorithms [21].”
Poisoning the Unlabeled Dataset of Semi-Supervised Learning (2021, methodology): “injecting unlabeled data to indiscriminately reduce model accuracy [6, 7].”
A Backdoor Attack Against LSTM-Based Text Classification Systems (2019, background): “Similar poisoning strategies which target other traditional machine learning models have also been developed, such as regression models [17] and clustering models [18].”
Data Poisoning Attacks Against Federated Learning Systems (2020, methodology): “Several poisoning attacks were developed for popular ML models including SVM [6,12,44,45,50,52], regression [19], dimensionality reduction [51], linear classiﬁers [12,23,57], unsupervised learning [7], and more recently, neural networks [12,30,42,45,53,58].”
On the Security of Machine Learning in Malware C&C Detection (2016, methodology): “For example, the technique used by Biggio et al. [Biggio et al. 2013], while only tested using SVM and neural networks, is theoretically applicable to any classiﬁer with a differentiable discriminant function.”
Cracking Classifiers for Evasion: A Case Study on the Google's Phishing Pages Filter (2016, background): “Such attacks will cause the learner to misclassify security violations (i.e., false negatives) [17][18][19][20].”