A robust hybrid of lasso and ridge regression

By Art B. Owen. 2006

302 citations recorded by the source corpus.

Subfields: Environmental Engineering, Management Science and Operations Research, Statistics and Probability, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Environmental Engineering

In Management Science and Operations Research

In Statistics and Probability

In Statistical Learning Theory

Citations in the corpus

Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning (2020, methodology): “We use a reverse Huber loss function for training [83, 97].”
Sever: A Robust Meta-Algorithm for Stochastic Optimization (2018, methodology): “Huberization [Owe07]), but as [KSL18] show (and we confirm in our experiments), correlated outliers often have lower loss than the real data under the learned model.”
Deeper Depth Prediction with Fully Convolutional Residual Networks (2016, positive): “Although this produces good results in our test cases, we found that using the reverse Huber (berHu) [25, 40] as loss function B yields a better final error than L2.”
Perspective Functions: Proximal Calculus and Applications in High-Dimensional Statistics (2016, methodology): “The scaled Lasso [26], a robust hybrid of ridge and Lasso regression [23], and the TREX [19] are important instances.”
Characterizing signal propagation to close the performance gap in unnormalized ResNets (2021, methodology): “We take the features from the ﬁnal residual stage and feed them into the up-projection blocks from (Silberman et al., 2012), then train with a reverse Huber loss (Laina et al., 2016; Owen, 2007).”
Perspective Functions: Properties, Constructions, and Examples (2016, background): “Applications of the perspective of the Berhu function in robust regression can be found in [40] and in [53].”
Choosing the Sample with Lowest Loss makes SGD Robust (2020, background): “Another line of recent work considers robustness in the high-dimensional setting ([27, 36, 8, 3, 24]) In this version, our focus is on general stochastic optimization in presence of outliers.”
Combinatorial Penalties: Structure preserved by convex relaxations (2017, methodology): “This penalty, called “Berhu" penalty [31], was introduced to produce a robust ridge regression estimator and was shown to be the convex envelope of F card 2 in [23].”
Least angle and ℓ1 penalized regression: A review (2008, methodology): “…that LASSO has had little impact on statistical practice, due to the inefficiency of the original LASSO and complexity of more recent algorithms (Osborne et al., 2000a); they add that this “efficient, simple algorithm for the LASSO as well as algorithms for stagewise regression and the new…”
Elastic-net regularization in learning theory (2008, methodology): “The method allows to select groups of correlated features when the groups are not known in advance (algorithms to enforce group-sparsity with preassigned groups of variables have been proposed in e.g. [31, 42, 22] using other types of penalties).”
Efficient Online and Batch Learning Using Forward Backward Splitting (2009, unknown): “Berhu regularization (Owen, 2006) is defined as r(w) = λ n ∑ j=1 b(w j ) = λ n ∑ j=1 |w j | [[|w j | ≤ γ]] + w 2 j + γ 2 2γ [[|w j | > γ]] .”
Fast Scene Understanding for Autonomous Driving (2017, methodology): “estimation branch uses the reverse Huber loss (berHu) [17],”
Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image (2017, background): “Another common choice is the Reversed Huber (denoted as berHu) loss function [27], defined as”
Group Regularized Estimation Under Structural Hierarchy (2014, background): “The same conclusion holds for the “`1 + `2” type penalties including the elastic net (Zou and Hastie, 2005) and Berhu (Owen, 2007).”