PAC-Bayesian Stochastic Model Selection

By David A. McAllester. 2003

321 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Management Science and Operations Research, Probabilistic & Bayesian Methods, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Probabilistic & Bayesian Methods

In Statistical Learning Theory

Citations in the corpus

PAC-Bayesian Theory Meets Bayesian Inference (2016, positive): “Since its early beginning [32, 47], the PAC-Bayesian theory claims to provide “PAC guarantees to Bayesian algorithms” (McAllester [32]).”
Tighter PAC-Bayes Bounds (2006, background): “The tightest bounds for practical applications appear to be the PAC-Bayes bound [4, 5].”
Exact minimax risk for linear least squares, and the lower tail of sample covariance matrices (2019, methodology): “…an exponential upper bound on Σ n op , which does not hold for heavy-tailed distributions. k k Instead, as in [Oli16], we use the so-called PAC-Bayesian inequality for empirical processes [McA99, McA03, LST03, Cat07, AC11], which is based on a variational representation of the relative entropy.”
PAC-Bayes Risk Bounds for Stochastic Averages and Majority Votes of Sample-Compressed Classifiers (2007, methodology): “If we limit ourselves to loss functions taking values in the [0,1] interval (including the loss value for abstaining), we can use the risk bound of Theorem 1 of McAllester (2003a).”
Supplementary Material to A PAC-Bayesian Approach for Domain Adaptation with Specialization to Linear Classiﬁers (2013, methodology): “First we recall the PAC-Bayesian bound proposed by McAllester (2003), which is stated without a term allowing to control the trade-oﬀ between the complexity and the risk.”
User-friendly introduction to PAC-Bayes bounds (2021, background): “The very first PAC-Bayes bounds by McAllester [134, 135] were empirical bounds.”
Fast Rates with Unbounded Losses (2016, methodology): “Thus, we bound the expected excess risk we get by randomizing over Π∣n in terms of a complexity term, which is just a variation of the term arising in standard PAC-Bayesian bounds (Catoni, 2003; McAllester, 2003), involving a KL divergence between ‘posterior’ and ‘prior’, which was first presented i”
On Bayesian bounds (2006, methodology): “The ﬁrst part of the result has earlier explicitly appeared in the literature (McAllester, 2003b, Lemma 8).”
Human Action Segmentation and Recognition Using Discriminative Semi-Markov Models (2011, methodology): “Our generalization analysis of the proposed approach is based on the PAC-Bayes theory introduced by McAllester and co-workers [12, 17–19].”
PAC-Bayesian learning of linear classifiers (2009, methodology): “The PAC-Bayes bound of McAllester (2003) can be obtained by using D(q, p) = 2(q - p)2 .”
Bayesian Gaussian process models : PAC-Bayesian generalisation error bounds and sparse approximations (2003, methodology): “The original formulation in [117] uses the Hoeffding bound (see Appendix A.”
Semi-supervised learning using randomized mincuts (2004, methodology): “mincuts, and also how this approach compares with the semi-supervised learning schemes of Zhu et al. (2003) and Joachims (2003).”
Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning) (2005, unknown): “The proof can be found in McAllester [2003]. The Kullback-Leibler (KL) divergence KL(q||p) is defined in section A.”
Structured Learning with Approximate Inference (2007, background): “We argue that without understanding combinations of inference and learning, such as these, that are appropriately compatible, learning performance under approximate inference cannot be guaranteed.”
Theory of classification : a survey of some recent advances (2005, methodology): “1 was derived by McAllester [163] and later extended in [164, 165].”
Generalization Bounds of SGLD for Non-convex Learning: Two Theoretical Viewpoints (2017, methodology): “To obtain data-dependent and algorithm-dependent bounds, we adopt two theoretical tools: uniform stability (Elisseeﬀ et al., 2005; Rakhlin et al., 2005) and PAC-Bayesian theory (McAllester, 2003; Germain et al., 2016).”
PAC-Bayes-Empirical-Bernstein Inequality (2013, methodology): “We start with providing the problem setting and then give some background on PAC-Bayesian analysis.”
PAC-Bayesian Generalisation Error Bounds for Gaussian Process Classification (2003, unknown): “The term “stochastic model selection” in (McAllester, 2002) refers to the probabilistic choice made by the Gibbs classifier on every test point.”