Simplified PAC-Bayesian Margin Bounds

By David A. McAllester. 2003

245 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Smoothness, Low Noise and Fast Rates (2010, methodology): “Improved margin bounds of the above form have been previously shown speciﬁcally for linear prediction in a Hilbert space (as in Support Vector Machines) based on the PAC Bayes theorem [20, 15].”
A PAC-Bayesian bound for Lifelong Learning (2013, methodology): “For each task we use 2% of the data (approximately 20 images) available for collie class and the same amount of images from the another task, such that data between different tasks does not overlap.”
Exploring Generalization in Deep Learning (2017, methodology): “…of the training data, with probability at least 1 − δ over the draw of the training data, the expected error of f w + ν can be bounded as follows [14]: size of traning set Figure 2: Sharpness and PAC-Bayes measures on a VGG network trained on subsets of CIFAR10 dataset with true or random labels.”
A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks (2017, methodology): “The lemma is analogous to similar analysis of Langford and Shawe-Taylor [7] and McAllester [8] obtaining PAC-Bayes margin bounds for linear predictors.”
Human Action Segmentation and Recognition Using Discriminative Semi-Markov Models (2011, methodology): “Our generalization analysis of the proposed approach is based on the PAC-Bayes theory introduced by McAllester and co-workers (Langford and McAllester 2004, McAllester 1998, McAllester 2003a, 2003b ).”
Learning to share visual appearance for multiclass object detection (2011, methodology): “We now develop a connection between regret and risk bounds via a PAC-Bayesian analysis (McAllester, 2003; Audibert & Bousquet, 2007; Catoni, 2007).”
Learning Kernel-Based Halfspaces with the 0-1 Loss (2011, background): “It was shown [5, 21] that m = Θ(log(1/δ)/(μ )(2)) examples are sufficient (and necessary) to learn a classifier for which Equation (4.”
PAC-Bayes Risk Bounds for Stochastic Averages and Majority Votes of Sample-Compressed Classifiers (2007, methodology): “If we limit ourselves to loss functions taking values in the [0,1] interval (including the loss value for abstaining), we can use the risk bound of Theorem 1 of McAllester (2003a).”
Information-Theoretic Local Minima Characterization and Regularization (2019, methodology): “Similar to Langford & Caruana (2002), Harvey et al. (2017) and Neyshabur et al. (2017), we apply the PAC-Bayes Theorem (McAllester, 2003) to derive a generalization bound for our metric.”
PAC-Bayes-Bernstein Inequality for Martingales and its Application to Multiarmed Bandits (2011, methodology): “…was applied to derive generalization bounds and new algorithms for linear classifiers and maximum margin methods (Langford and Shawe-Taylor, 2002; McAllester, 2003; Germain et al., 2009), structured prediction (McAllester, 2007), and clustering-based classification models (Seldin and Tishby,…”
Learning Efficient Random Maximum A-Posteriori Predictors with Non-Decomposable Loss Functions (2013, background): “The perturb-max models in Equation (3) play a key role in PAC-Bayesian theory [14, 11, 19, 3, 20, 5, 10].”
PAC-Bayesian Analysis of Co-clustering and Beyond (2010, methodology): “…have been applied in the analysis of decision trees (Mansour and McAllester, 2000), Support Vector Machines (SVMs) (Langford and Shawe-Taylor, 2002; McAllester, 2003; Langford,2005; Ambroladze et al., 2007; Crammer et al., 2009; Germain et al., 2009), transductive learning (Derbeko et al., 2004),…”
On Bayesian bounds (2006, methodology): “The ﬁrst part of the result has earlier explicitly appeared in the literature (McAllester, 2003b, Lemma 8).”
Trading Value and Information in MDPs (2012, methodology): “We begin by recalling the PAC-Bayesian bound [6].”