Resilient Linear Classification: An Approach to Deal with Attacks on Training Data

By Sangdon Park, James Weimer, Insup Lee. 2017

8 citations recorded by the source corpus.

Data-driven techniques are used in cyber-physical systems (CPS) for controlling autonomous vehicles, handling demand responses for energy management, and modeling human physiology for medical devices.These data-driven techniques extract models from training data, where their performance is often analyzed with respect to random errors in the training data. However, if the training data is maliciously altered by attackers, the effect of these attacks on the learning algorithms underpinning data-driven CPS have yet to be considered. In this paper, we analyze the resilience of classification algorithms to training data attacks. Specifically, a generic metric is proposed that is tailored to measure resilience of classification algorithms with respect to worst-case tampering of the training data. Using the metric, we show that traditional linear classification algorithms are resilient under restricted conditions.To overcome these limitations, we propose a linear classification algorithm with a majority constraint and prove that it is strictly more resilient than the traditional algorithms.Evaluations on both synthetic data and a real-world retrospective arrhythmia medical case-study show that the traditional algorithms are vulnerable to tampered training data, whereas the proposed algorithm is more resilient (as measured by worst-case tampering).

Subfields: Artificial Intelligence, Control and Systems Engineering, Software

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Percy Liang
2. Jacob Steinhardt
3. Pang Wei Koh
4. Zhiyi Tian
5. Lei Cui
6. Shui Yu
7. Jie Liang
8. Yejin Choi

In Control and Systems Engineering

1. Percy Liang
2. Jacob Steinhardt

In Software

1. Percy Liang
2. Jacob Steinhardt

Citations in the corpus

• Certified Defenses for Data Poisoning Attacks (2017, background): “A defender who trains naïvely on the full (clean + poisoned) data D c ∪ D p is doomed to failure, as even a single poisoned point can in some cases arbitrarily change the model (Liu and Zhu, 2016; Park et al., 2017).”
• A Comprehensive Survey on Poisoning Attacks and Countermeasures in Machine Learning (2022, background): “[95] proposed a metric for quantifying the resilience of learning algorithms to assist the design of resilient learning algorithms.”