Mean Estimation and Regression Under Heavy-Tailed Distributions: A Survey

By Gábor Lugosi, Shahar Mendelson. Foundations of Computational Mathematics, 2019

306 citations recorded by the source corpus.

We survey some of the recent advances in mean estimation and regression function estimation. In particular, we describe sub-Gaussian mean estimators for possibly heavy-tailed data in both the univariate and multivariate settings. We focus on estimators based on median-of-means techniques, but other methods such as the trimmed-mean and Catoni’s estimators are also reviewed. We give detailed proofs for the cornerstone results. We dedicate a section to statistical learning problems—in particular, regression function estimation—in the presence of possibly heavy-tailed data.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/95d4f21a498487f92fcb34d4040d39876a179105

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Exact minimax risk for linear least squares, and the lower tail of sample covariance matrices (2019, background): “On the other hand, expected risk only provides limited information on the tails of the risk in the high-conﬁdence regime: in the case of heavy-tailed noise, the OLS estimator may perform poorly, and dedicated robust estimators may be required (see, e.g., [AC11] and the references in [LM19]).”
An approximate sampler for energy-based models with divergence diagnostics (2022, background): “However, in order to provide guarantees about the accuracy of these estimates — e.g. in terms of confidence bounds — one would need to estimate these variances, or their (1 + α )-moment for some α > 0 as discussed by Lugosi and Mendelson (2019, p. 8).”
Private Mean Estimation of Heavy-Tailed Distributions (2020, unknown): “A recent series of papers has focused on identifying effective methods and making them computationally efﬁcient [LM19b, Hop18, CFB19, DL19, LM19a].”
Robust multivariate mean estimation: The optimality of trimmed mean (2019, background): “Once again, we refer to the survey [20] for related results.”