Some PAC-Bayesian Theorems

By David A. McAllester. 1998

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistical Learning Theory

Citations in the corpus

PAC-Bayesian Theory Meets Bayesian Inference (2016, positive): “Since its early beginning [32, 47], the PAC-Bayesian theory claims to provide “PAC guarantees to Bayesian algorithms” (McAllester [32]).”
Wireless Network Intelligence at the Edge (2018, methodology): “Utilizing Bayesian learning methods, one can resolve these issues, resulting in the following PAC-Bayes bound [135].”
Predictive Multiplicity in Classification (2019, methodology): “…explicitly represent uncertainty over the set of competing models by means of a posterior probability distribution, but then use the posterior distribution to construct a single model for deployment [e.g., via a majority vote or a randomization procedure as in PAC Bayesian approaches, 18, 32].”
A PAC-Bayesian bound for Lifelong Learning (2013, methodology): “aining data and particular learning algorithm. In this work we retain Baxter’s model of lifelong learning but apply PAC-Bayesian theory to analyze it. PAC-Bayesian theory was introduced by McAllester [8] and combines advantages of PAC and 1 Bayes frameworks. It allows to obtain bounds that are data ”
Exploring Generalization in Deep Learning (2017, background): “The PAC-Bayesian framework [16, 17] provides guarantees on the expected error of a randomized predictor (hypothesis), drawn form a distribution denotedQ and sometimes referred to as a “posterior” (although it need not be the Bayesian posterior), that depends on the training data.”
A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks (2017, background): “The PAC-Bayesian framework [9, 10] provides generalization guarantees for randomized predictors, drawn form a learned distribution Q (as opposed to a learned single predictor) that depends on the training data.”
Stronger generalization bounds for deep nets via a compression approach (2018, methodology): “Using PAC-Bayes bound [McAllester, 1998, 1999] this noise stability yielded generalization bounds for fully connected nets of depth 2.”
Curriculum learning of multiple tasks (2014, methodology): “We study this question by using PAC-Bayesian theory [24] to prove a generalization bound that depends on the data representation and algorithm used to solve the tasks.”
PAC-Bayes-Bernstein Inequality for Martingales and its Application to Multiarmed Bandits (2011, methodology): “The PAC-Bayesian analysis was introduced over a decade ago (Shawe-Taylor and Williamson, 1997; Shawe-Taylor et al., 1998; McAllester, 1998; Seeger, 2002) and has since made a significant contribution to the analysis and development of supervised learning methods.”
Information-Theoretic Generalization Bounds for SGLD via Data-Dependent Estimates (2019, methodology): “We make repeated use of generalized notions of priors and posteriors, which arise in the PAC-Bayes literature ([7, 20, 30], etc.”
Deep neural networks are biased towards simple functions (2018, background): “…PAC-Bayesian generalization bounds, which apply to nondeterministic learning algorithms and depend on the probability distribution of the weights that the algorithm returns given the training set [McAllester, 1999,Catoni, 2007, Lever et al., 2013, Dziugaite and Roy, 2018, Neyshabur et al., 2017b].”
PAC-Bayesian Bounds based on the Rényi Divergence (2016, methodology): “Likewise, most PAC-Bayesian bounds on the risk of the Gibbs classiﬁer depend on the KL divergence [ e.g. , McAllester, 1999, Langford and Shawe-Taylor, 2002, Seeger, 2003].”
Chromatic PAC-Bayes Bounds for Non-IID Data (2009, background): “Pac -Bayes bounds, introduced by McAllester (1999), and reﬁned in several occasions (Seeger, 2002a; Langford, 2005; Audibert and Bousquet, 2007), are some of the most appealing advances from the tightness and accuracy points of view (an excellent monograph on the Pac -Bayesian framework is that of…”
PAC-Bayesian bounds for randomized empirical risk minimizers (2007, unknown): “The PAC-Bayesian point of view, introduced in the context of classiﬁcation by McAllester [16, 17] is based on the following remark: while classical measures of complexity (like VC-dimension) require theoretical results on the submodels, the introduction of a probability measure π on the model Θ allo”
Understanding Generalization through Visualizations (2019, unknown): “PAC-Bayes bounds (McAllester, 1998, 1999; Neyshabur et al., 2017; Bartlett and Mendelson, 2003; Neyshabur et al., 2015; Golowich et al., 2018), provide generalization guarantees for randomized predictors drawn from a learned distribution that depends on the training data, as opposed to a learned…”
User-friendly introduction to PAC-Bayes bounds (2021, background): “The very first PAC-Bayes bounds by McAllester [134, 135] were empirical bounds.”
Regret Bounds for Lifelong Learning (2016, background): “At each time t ∈ {1, . . . , T}, a task is presented to the learner in the following manner: 1. nature choses Pt , no assumption is made on this choice.”
Human Action Segmentation and Recognition Using Discriminative Semi-Markov Models (2011, methodology): “Our generalization analysis of the proposed approach is based on the PAC-Bayes theory introduced by McAllester and co-workers [12, 17–19].”