Data Poisoning Attacks against Autoregressive Models

By Scott Alfeld, Xiaojin Zhu, Paul R. Barford. 2016

265 citations recorded by the source corpus.

Forecasting models play a key role in money-making ventures in many different markets. Such models are often trained on data from various sources, some of which may be untrustworthy.An actor in a given market may be incentivised to drive predictions in a certain direction to their own benefit.Prior analyses of intelligent adversaries in a machine-learning context have focused on regression and classification.In this paper we address the non-iid setting of time series forecasting.We consider a forecaster, Bob, using a fixed, known model and a recursive forecasting method.An adversary, Alice, aims to pull Bob's forecasts toward her desired target series, and may exercise limited influence on the initial values fed into Bob's model.We consider the class of linear autoregressive models, and a flexible framework of encoding Alice's desires and constraints.We describe a method of calculating Alice's optimal attack that is computationally tractable, and empirically demonstrate its effectiveness compared to random and greedy baselines on synthetic and real-world time series data.We conclude by discussing defensive strategies in the face of Alice-like adversaries.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion (2020, background): “A data poisoning [1,9,13,25,31,51,54, 67] attack modifies the training data.”
Towards the Science of Security and Privacy in Machine Learning (2016, background): “Beyond classiﬁcation: While most work has focused on attacking classiﬁers, Alfeld et al. [54] look at autoregressive models.”
Attack of the Tails: Yes, You Really Can Backdoor Federated Learning (2020, background): “However, it can also be used to force the learner into a nefarious model [62, 63, 64].”
Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning (2017, methodology): “After presenting the details of the threat model above, we emphasize again the differences between this work and a long line of existing work studying poisoning attacks and defenses [10], [38], [12], [11], [68], [44], [9], [35], [70], [39], [61].”
Contamination Attacks and Mitigation in Multi-Party Machine Learning (2018, background): “In this way our work is similar to work on targeted poison attacks [2, 4, 21, 23, 42, 43] in machine learning, where the aim is to degrade the performance of a model.”
Data Poisoning Attacks on Factorization-Based Collaborative Filtering (2016, methodology): “Similar techniques were later generalized to optimize data poisoning attacks for several other important learning algorithms, such as Lasso regression [7], topic modeling [8], and autoregressive models [17].”
Adversarial Examples: Attacks and Defenses for Deep Learning (2017, background): “, training data poisoning) is another interesting topic and has been studied in [27]–[32].”
SoK: Security and Privacy in Machine Learning (2018, background): “White-Box Full No [42], [43], [44] [24], [21], [44], [45], [46] [47] [48]”
Teacher Improves Learning by Selecting a Training Subset (2018, background): “Machine teaching applications include education [14, 33, 39, 27, 13, 34], computer security [2, 1, 32], and interactive machine learning [40, 12, 24].”
A Survey on Security Threats and Defensive Techniques of Machine Learning: A Data Driven View (2018, background): “The poisoning attack is a type of causative attack, which disrupts the availability and the integrity of machine learning models via injecting adversarial samples to the training data set [36], [37].”
Hu-Fu: Hardware and Software Collaborative Attack Framework Against Neural Networks (2018, background): “Another type of attack is called data poisoning [26], [27].”
Secure and Robust Machine Learning for Healthcare: A Survey (2020, background): “Moreover, ML/DL models have been found strictly vulnerable to various security and privacy threats such as adversarial attacks [16], model [88] and data poisoning attacks [89], etc.”
Online Data Poisoning Attack (2019, background): “A variety of attacks against other learners have been developed, including neural networks [20, 12], autoregressive models [7, 1], linear and stochastic bandits [17, 11], collaborative filtering [14], and sentiment analysis [21], etc.”
Explicit Defense Actions Against Test-Set Attacks (2017, methodology): “She aims to perform an attractive (Alfeld, Zhu, and Barford 2016) attack, moving Bob’s prediction towards some target.”
Invisible Backdoor Attacks on Deep Neural Networks Via Steganography and Regularization (2019, background): “However, this is a double-edged sword, as this power can also be easily affected by slight perturbations, such as evasion attacks [16] and poisoning attacks [17, 18, 19].”
Robust Linear Regression Against Training Data Poisoning (2017, methodology): “Similar poisoning attack techniques have also been generalized to other widely used learning algorithms, such as Lasso regression [29], topic modeling [22], and autoregressive models [1].”
Security Risks in Deep Learning Implementations (2017, background): “Data poisoning attacks[1, 3] may not rely on software vulnerabilities.”
An Overview of Machine Teaching (2018, background): “• In one type of adversarial attack known as training-set poisoning, an attacker manipulates the behavior of a machine learning system by maliciously modifying the training data [2,30,31].”