PAC-bayes bounds with data dependent priors

By Emilio Parrado-Hernández, Amiran Ambroladze, John Shawe-Taylor, Shiliang Sun. 2012

140 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Theory and Mathematics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Pierre Alquier
2. Christoph H. Lampert
3. Anastasia Pentina
4. Ben London
5. Ted Sandler
6. Pascal Germain
7. Gintare Karolina Dziugaite
8. Jean-Francis Roy

In Computational Theory and Mathematics

1. Pierre Alquier
2. Ben London
3. Ted Sandler
4. Xin Xu
5. Ashish J. Khisti
6. Christoph H. Lampert
7. Shiliang Sun
8. François Laviolette

Citations in the corpus

• A PAC-Bayesian bound for Lifelong Learning (2013, background): “Parrado-Hernández et al. (2012) showed that priors can be learned by splitting the available training data into two parts, one for learning a prior, one for learning the predictor.”
• PAC-Bayesian Bounds based on the Rényi Divergence (2016, methodology): “So far, most algorithms derived from PAC-Bayesian bounds are KL-regularized [ e.g. Germain et al., 2009, Parrado-Hernández et al., 2012, Pentina and Lampert, 2015, Alquier et al., 2015].”
• Bayesian Counterfactual Risk Minimization (2018, methodology): “This type of prior, known as distribution-dependent, was introduced by Catoni [4] and later developed by others [17, 23, 24, 9] to obtain tight PAC-Bayesian bounds.”
• A survey of multi-view machine learning (2013, background): “However, the tightest bounds so far for practical applications appear to be the PAC-Bayes bound [25, 26] for which the most recent research outcome is using data-dependent priors [30].”
• Information-Theoretic Generalization Bounds for SGLD via Data-Dependent Estimates (2019, methodology): “The idea of using data-dependent priors to obtain tighter bounds is standard in the PAC-Bayes literature [2, 10, 22, 27], but its utility in the present work is brought through by our introduction of data-dependent estimates.”
• User-friendly introduction to PAC-Bayes bounds (2021, methodology): “2 Bounds with data-dependent priors To use data in order to improve the prior is actually an old idea: we found such approaches in [173, 41, 42, 206, 43, 154, 120, 70, 71, 69].”
• Multi-view learning overview: Recent progress and new challenges (2017, methodology): “PAC-Bayes analysis is a related technique for data-dependent theoretical analysis, which often gives tight generation bounds [38].”