Sharp Oracle Inequalities for Aggregation of Affine Estimators
By Arnak S. Dalalyan, Joseph Salmon. 2011
89 citations recorded by the source corpus.
We consider the problem of combining a (possibly uncountably infinite) set of affine estimators in non-parametric regression model with heteroscedastic Gaussian noise. Focusing on the exponentially weighted aggregate, we prove a PAC-Bayesian type inequality that leads to sharp oracle inequalities in discrete but also in continuous settings. The framework is general enough to cover the combinations of various procedures such as least square regression, kernel ridge regression, shrinking estimators and many other estimators used in the literature on statistical inverse problems. As a consequence, we show that the proposed aggregate provides an adaptive estimator in the exact minimax sense without neither discretizing the range of tuning parameters nor splitting the set of observations. We also illustrate numerically the good performance achieved by the exponentially weighted aggregate.
Subfields: Artificial Intelligence, Statistics and Probability
View Paper: https://www.semanticscholar.org/paper/15b8d0cc2b7d493f2124628dffd7aab421c6dc4a
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Statistics and Probability
Citations in the corpus
- Sharp oracle inequalities for Least Squares estimators in shape restricted regression (2015, unknown): “There exist procedures to aggregate with no sample splitting estimators of the form Ay where A is a deterministic n×nmatrix [22, 17, 16, 7], leading to sharp oracle inequalities.”
- User-friendly introduction to PAC-Bayes bounds (2021, background): “Many aggregates share a formal similarity with the EWA of online learning and with the Gibbs posterior of the PAC-Bayes approach, we refer the reader to [144, 106, 202, 139, 203, 42, 206, 119, 116, 64, 40, 183, 65, 60, 63, 59, 62, 126, 61].”