Tighter PAC-Bayes bounds through distribution-dependent priors

By Guy Lever, François Laviolette, John Shawe-Taylor. 2013

122 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Management Science and Operations Research, Statistics and Probability, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Pierre Alquier
2. Pascal Germain
3. François Laviolette
4. Jean-Francis Roy
5. Gintare Karolina Dziugaite
6. Daniel M. Roy
7. Alexandre Lacasse
8. Mario Marchand

In Management Science and Operations Research

1. Pierre Alquier
2. Pascal Germain
3. Mario Marchand
4. R. Meir
5. Ron Amit
6. Alexandre Lacasse
7. François Laviolette
8. Gintare Karolina Dziugaite

In Statistics and Probability

1. Pierre Alquier
2. Pascal Germain
3. Mario Marchand
4. R. Meir
5. Alexandre Lacasse
6. François Laviolette
7. Gintare Karolina Dziugaite
8. Jean-Francis Roy

In Statistical Learning Theory

1. Pierre Alquier
2. François Laviolette
3. Jean-Francis Roy
4. Pascal Germain
5. Mario Marchand
6. Alexandre Lacasse
7. R. Meir
8. Ron Amit

Citations in the corpus

• PAC-Bayesian Theory Meets Bayesian Inference (2016, neutral): “However, despite the amount of work dedicated to this statistical learning theory—many authors improved the initial results [9, 29, 33, 43, 48] and/or generalized them for various machine learning setups [4, 14, 20, 28, 38, 44, 45, 46]—it is mostly used as a frequentist method.”
• User-friendly introduction to PAC-Bayes bounds (2021, methodology): “2 Bounds with data-dependent priors To use data in order to improve the prior is actually an old idea: we found such approaches in [173, 41, 42, 206, 43, 154, 120, 70, 71, 69].”
• Risk bounds for the majority vote: from a PAC-Bayesian analysis to a learning algorithm (2015, background): “There have already been some attempts to develop PAC-Bayesian bounds that do not rely on the KL-divergence (see the localized Priors of Catoni (2007), or the distributiondependent priors of Lever et al. (2013)).”
• Meta-Learning by Adjusting Priors Based on Extended PAC-Bayes Theory (2017, background): “The PAC-Bayes framework has been widely studied in recent years, and has given rise to significant flexibility in learning, and, more importantly, to some of the best generalization bounds available Seeger (2002); Catoni (2007); Audibert (2010); Lever et al. (2013).”
• Entropy-SGD optimizes the prior of a PAC-Bayes bound: Data-dependent PAC-Bayes priors via differential privacy (2017, background): “Lever, Laviolette, and Shawe-Taylor (2013) extend this approach to generalization bounds and consider both data-independent and data-dependent choices for P .”