Outlier Detection Using Nonconvex Penalized Regression

By Yiyuan She, Art B. Owen. arXiv.org, 2010

338 citations recorded by the source corpus.

This article studies the outlier detection problem from the standpoint of penalized regression. In the regression model, we add one mean shift parameter for each of the n data points. We then apply a regularization favoring a sparse vector of mean shift parameters. The usual L1 penalty yields a convex criterion, but fails to deliver a robust estimator. The L1 penalty corresponds to soft thresholding. We introduce a thresholding (denoted by Θ) based iterative procedure for outlier detection (Θ–IPOD). A version based on hard thresholding correctly identifies outliers on some hard test problems. We describe the connection between Θ–IPOD and M-estimators. Our proposed method has one tuning parameter with which to both identify outliers and estimate regression coefficients. A data-dependent choice can be made based on the Bayes information criterion. The tuned Θ–IPOD shows outstanding performance in identifying outliers in various situations compared with other existing approaches. In addition, Θ–IPOD is much faster than iteratively reweighted least squares for large data, because each iteration costs at most O(np) (and sometimes much less), avoiding an O(np2) least squares estimate. This methodology can be extended to high-dimensional modeling with p ≫ n if both the coefficient vector and the outlier pattern are sparse.

Subfields: Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/3accd0b572e3267e6b2832c70ac8c4aa92064f5c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

A General Method for Robust Bayesian Modeling (2015, methodology): “This is the idea behind She and Owen (2011) (for linear regression) and Tib-shirani and Manning (2013) (for logistic regression).”
Radar: Residual Analysis for Anomaly Detection in Attributed Networks (2017, methodology): “Residual analysis [She and Owen, 2011], which is initiated to study the residuals between true data and estimated data for regression problems, is able to help us understand anomalies generally.”
Robust Sparse Regression under Adversarial Corruption (2013, background): “Another modification of Lasso accounts for the corruption in the response via an additional variable z (Laska et al., 2009; Li, 2011; She & Owen, 2010): min β,z ‖Xβ − y − z‖22 + λ ‖β‖1 + γ ‖z‖1 .”
Interestingness Prediction by Robust Learning to Rank (2014, background): “2 For a thorough discussion from a statistical perspective, please read [9,10,8,38].”
Moving Object Detection by Detecting Contiguous Outliers in the Low-Rank Representation (2011, background): “In order to go beyond the ‘1-penalty, nonconvex penalties have been explored in recent literature [52], [54].”
Linear regression with sparsely permuted data (2017, background): “In view of the fact that the optimization problem (3) is not computationally tractable, we consider a convex relaxation that takes the form of a robust regression estimation problem as it has been considered before in different contexts [27, 40, 50], with the”
Robust linear regression: A review and comparison (2014, positive): “She and Owen (2011) showed that the proposed estimates of (2.22) with hard or SCAD penalties are equivalent to the M-estimates with certain redescending ψ functions and thus will be resistent to high leverage outliers if a high breakdown point robust estimates are used as the initial values.”
Fast and Flexible ADMM Algorithms for Trend Filtering (2014, methodology): “To simultaneously estimate a trend and detect outliers, we solve (ˆ β, ˆ z ) = argmin as in Kim et al. (2009), She & Owen (2011), where the nonzero components of ˆ z correspond to the adaptively detected outliers.”
Robust Regression via Hard Thresholding (2015, neutral): “1805.”