PAC-Bayesian bounds for randomized empirical risk minimizers

By Pierre Alquier. 2007

62 citations recorded by the source corpus.

The aim of this paper is to generalize the PAC-Bayesian theorems proved by Catoni [6, 8] in the classification setting to more general problems of statistical inference. We show how to control the deviations of the risk of randomized estimators. A particular attention is paid to randomized estimators drawn in a small neighborhood of classical estimators, whose study leads to control of the risk of the latter. These results allow us to bound the risk of very general estimation procedures, as well as to perform model selection.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/15d3e30305bd4b89bfb784feefd0a1a633beeca5

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

ROBUST EMPIRICAL MEAN ESTIMATORS (2011, methodology): “Catoni [12, 14] has recently developed a Pac-Bayesian approach that has applied in several problems of non parametric statistics including linear regression and classiﬁcation [1, 2, 13].”
Challenging the empirical mean and empirical variance: a deviation study (2010, methodology): “ce of estimators. Our deviation study will use two kinds of tools: M-estimators to truncate observations and PAC-Bayesian theorems to combine estimates on the same sample without using a split scheme [9, 8, 10, 5, 2, 1]. Its general conclusion is that, whereas the deviations of the empirical mean es”
A general framework for updating belief distributions (2013, background): “While this methodology has made rapid developments in recent years, including the development of sampling algorithms, for complex data structures there are still issues about just how large the supports are and indeed how complicated inference can be and how to construct priors which capture…”
User-friendly introduction to PAC-Bayes bounds (2021, methodology): “This truncation argument is reused in [2] but only the oracle bounds are provided there.”