Optimal Rates of Aggregation

By Alexandre B. Tsybakov. Annual Conference Computational Learning Theory, 2003

313 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/be9f68f3d90d9ea0a283c1794f37c4e30b40b71b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

Sharp Oracle Inequalities for Aggregation of Affine Estimators (2011, methodology): “Dalalyan and Tsybakov [19, 20] showed the result of [44] remains valid under more general (non-Gaussian) noise distributions and when the constituent estimators are independent of the data used for the aggregation.”
Communication trade-offs for Local-SGD with large step size (2019, background): “5 α 1 , matches the statistical minimax rate [48] for least squares regression: from the statistical point of view, if we are only given NP independent observations, then no estimator can have an error uniformly lower than σ 2 ( PN ) − 1 .”
Model selection via testing: an alternative to (penalized) maximum likelihood estimators (2006, methodology): “Let us just mention here a few key references on the subject of aggregation like Juditsky and Nemirovski [37], Nemirovski [50], Yang ([66] [67] [68] and [70]), Catoni ([23] and [24]), Tsybakov [56] and Wegkamp [63].”
ℓ1-regularized linear regression: persistence and oracle inequalities (2011, background): “Tsybakov showed [38] that this rate cannot be improved, and that a complex estimator that extends ideas of Catoni [9] achieves this rate.”
Adapting to Unknown Smoothness by Aggregation of Thresholded Wavelet Estimators (2006, background): “The aggregation framework considered, among others, by [34], [54], [13],[46], [49], [5], [6] is the following: take F0 a finite subset of F , our aim is to mimic (up to an 6”
Sparse Modeling for Image and Vision Processing (2014, background): “‖22 ≈ σ2p m , which is attained for ordinary leastsquares, and is the best possible [Tsybakov, 2003].”
Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity (2008, background): “In other words, we deal with the problems of model selection (MS) type aggregation or linear/convex aggregation respectively [27, 31].”
Recursive Aggregation of Estimators by the Mirror Descent Algorithm with Averaging (2005, unknown): “This aggregation problem is similar to those considered, for instance, in [15] and [31] for the regression model with squared loss.”
Learning by mirror averaging (2005, background): “Lower bounds can be established showing that, under some assumptions, the smallest possible value of ∆ n,M in a minimax sense has the form with some constant C > 0; cf. [24].”
Sparsity in penalized empirical risk minimization (2009, background): “Aggregation techniques, especially in the case of L2-regression, have been introduced in [23] and [28] and studied extensively in [4,5,9,26,29] among others.”
SPADES AND MIXTURE MODELS (2009, background): “ approximately as good as the best among f 1,...,fM or approximately as good as the best linear or convex combination of f 1,...,fM. General notions of aggregation and optimal rates are introduced in [33, 40]. Aggregation of density estimators is discussed in [35, 36, 38] and more recently in [5] wh”
Optimal rates and adaptation in the single-index model using aggregation (2007, methodology): “…we obtain where ˜ f ( y (1) where the inﬁmum is taken among all estimators R → R based on data from model (1.1) with d = 1 (univariate regression). order In to bound (5.5) from below, we use the following Theorem, from Tsybakov (2003a), which is a standard tool for the proof of such a lower bound.”
Optimal Statistical Rates for Decentralised Non-Parametric Regression with Linear Speed-Up (2019, background): “We consider the case γ > 0 which does not include the finitedimensional setting H = R, γ = 0, where the optimal rate is O(d/(nm)) [54].”
Exact minimax risk for linear least squares, and the lower tail of sample covariance matrices (2019, background): “. 9 Since n− d+ 1 > n, Corollary 1 implies a lower bound of σ2d/n. The minimax risk for linear regression has been determined under some additional boundedness assumptions on Y (and thus on β∗) by [Tsy03], showing that it scales as Θ(σ2d/n) up to numerical constants. The proof of the lower bound ”
Time-uniform central limit theory and asymptotic confidence sequences (2021, methodology): “Here we will consider the general approach of stacking introduced by Breiman [5] and further studied by Tsybakov [54] and van der Laan et al.”
Semiparametric Counterfactual Density Estimation (2021, methodology): “(14) for some appropriate selection set, e.g., the standard simplex B = {(β1, ..., βK) ∈ RK : βk ≥ 0, ∑ k βk = 1} for convex aggregation [Rigollet and Tsybakov, 2007, Tsybakov, 2003].”