Robust High-Dimensional Linear Regression

By Chang Liu, Bo Li, Yevgeniy Vorobeychik, Alina Oprea. 2016

The effectiveness of supervised learning techniques has made them ubiquitous in research and practice. In high-dimensional settings, supervised learning commonly relies on dimensionality reduction to improve performance and identify the most important factors in predicting outcomes. However, the economic importance of learning has made it a natural target for adversarial manipulation of training data, which we term poisoning attacks. Prior approaches to dealing with robust supervised learning rely on strong assumptions about the nature of the feature matrix, such as feature independence and sub-Gaussian noise with low variance. We propose an integrated method for robust regression that relaxes these assumptions, assuming only that the feature matrix can be well approximated by a low-rank matrix. Our techniques integrate improved robust low-rank matrix approximation and robust principle component regression, and yield strong performance guarantees. Moreover, we experimentally show that our methods significantly outperform state of the art both in running time and prediction error.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Xinyun Chen
2. Bo Li
3. Dawn Song
4. Chang Liu
5. Nicholas Carlini
6. Kimberly Lu
7. Percy Liang
8. Yoshua Bengio

Citations in the corpus

• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning (2017, background): “After presenting the details of the threat model above, we emphasize again the differences between this work and a long line of existing work studying poisoning attacks and defenses [10], [38], [12], [11], [68], [44], [9], [35], [70], [39], [61].”