ROBUST EMPIRICAL MEAN ESTIMATORS

By Matthieu Lerasle, Roberto Imbuzeiro Oliveira. 2011

87 citations recorded by the source corpus.

We study robust estimators of the mean of a probability measure P, called robust empirical mean estimators. This elementary construction is then used to revisit a problem of aggregation and a problem of estimator selection, extending these methods to not necessarily bounded collections of previous estimators. We consider then the problem of robust M-estimation. We propose a slightly more complicated construction to handle this problem and, as examples of applications, we apply our general approach to least- squares density estimation, to density estimation with Kullback loss and to a non-Gaussian, unbounded, random design and heteroscedas- tic regression problem. Finally, we show that our strategy can be used when the data are only assumed to be mixing.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates (2018, methodology): “This idea is studied in Alon et al. (1999), Jerrum et al. (1986), Lerasle and Oliveira (2011), Minsker et al. (2015), Nemirovskii et al. (1983), and has been applied to bandit and least square regression problems (Bubeck et al., 2013, Kogler and Traxler, 2016, Lugosi and Mendelson, 2016) as well as…”
Geometric median and robust estimation in Banach spaces (2013, methodology): “(see Lemma 23 in [28] for a rigorous proof of this fact).”
Empirical risk minimization for heavy-tailed losses (2014, background): “e. Also, the median-of-means estimate is useful even when the variance is inﬁnite and only a moment of order 1+ǫexists for some ǫ>0 (see Bubeck, Cesa-Bianchi and Lugosi [11]). Lerasle and Oliveira [19] consider empirical minimization of the median-ofmeans estimator and obtain interesting results ”
Risk minimization by median-of-means tournaments (2016, background): “Devroye, Lerasle, Lugosi, and Oliveira [14] Hsu and Sabato [18], Lerasle and Oliveira [27], Minsker [34], Audibert and Catoni [3].”
Regularization, sparse recovery, and median-of-means tournaments (2017, methodology): “We refer to Audibert and Catoni [1], Hsu and Sabato [4], Lerasle and Oliveira [9], Minsker [17], Brownlees, Joly, and Lugosi [3], Lugosi and Mendelson [10] for a sample of the literature.”
Sub-Gaussian mean estimators (2015, unknown): “Recently, there has been a surge of interest in sub-Gaussian estimators, their generalizations to multivariate settings, and their applications in a variety of statistical learning problems where heavy-tailed distributions may be present, see, for example, Catoni [5], Hsu and Sabato [7], Brownlees, ”
Loss Minimization and Parameter Estimation with Heavy Tails (2013, unknown): “We have also since become aware of the earlier work by Lerasle and Oliveira (2011), which applies the median-of-means technique to empirical risks in various settings much like the way we do in Algorithm 3, although our metric formulation is more general.”
Sub-Gaussian estimators of the mean of a random matrix with heavy-tailed entries (2016, unknown): “[LO11] Matthieu Lerasle and Roberto I Oliveira.”
Distributed Statistical Estimation and Rates of Convergence in Normal Approximation (2017, unknown): “If Var(X) = σ2 ∞, it has been shown (for example, by Lerasle and Oliveira, 2011) that the median-of-means estimator θ̂(k) satisfies ∣∣∣θ̂(k) − θ∗∣∣∣ ≤ 2σ√6e√ log(1/α) N (3) with probability ≥ 1 − α if k = blog(1/α)c + 1.”
Robust estimation via robust gradient estimation (2018, methodology): “For the problem of one-dimensional mean estimation, Lerasle and Oliveira (2011) and Catoni (2012) proposed robust mean estimators that achieve exponential concentration around the true mean for any distribution with bounded second moment.”
Mean Estimation and Regression Under Heavy-Tailed Distributions: A Survey (2019, unknown): “A growing body of recent work has addressed the problem of constructing regression function estimators that work well even when some of the f (X) and Y may be heavy-tailed, see Audibert and Catoni [4], Brownlees et al. [11], Catoni and Giulini [15], Chichignoud and Lederer [19], Fan et al. [28], Hsu”
Sub-Gaussian estimators of the mean of a random vector (2017, background): “…multivariate case by replacing the median by its natural multivariate extension, the so-called “ge-ometric (or spatial) median” (i.e., the point that minimizes the sum of the Euclidean distances to the sample means within each block) see Lerasle and Oliveira [14], Hsu and Sabato [9], Minsker [16].”