PAC-Bayes & Margins

By J. Langford, John Shawe-Taylor. 2002

247 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistical Learning Theory

Citations in the corpus

PAC-Bayesian Theory Meets Bayesian Inference (2016, positive): “That is, the provided results express upper bounds on E f ∼ ˆ ρ L D ( f ) for any (bounded, sub-Gaussian, or sub-gamma) losses.”
Smoothness, Low Noise and Fast Rates (2010, methodology): “Improved margin bounds of the above form have been previously shown specifically for linear prediction in a Hilbert space based on the PAC Bayes theorem [19, 15].”
Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networks (2018, background): “This margin has been studied in relationship to generalization bounds [17, 3, 22].”
A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks (2017, methodology): “The lemma is analogous to similar analysis of Langford and Shawe-Taylor [7] and McAllester [8] obtaining PAC-Bayes margin bounds for linear predictors, and the proof, presented in Section 4, is essentially the same.”
Learning from Multiple Partially Observed Views - an Application to Multilingual Text Categorization (2009, unknown): “[9]).”
A New PAC-Bayesian Perspective on Domain Adaptation (2015, unknown): “Moreover, recall that Germain et al. (2013) choose to ignore the ρ -dependent non-constant term λ ( ρ ).”
Risk bounds for the majority vote: from a PAC-Bayesian analysis to a learning algorithm (2015, methodology): “Langford and Shawe-Taylor (2002) have shown that under some circumstances, the factor of two can be reduced to (1 + ).”
Sharp Oracle Inequalities for Aggregation of Affine Estimators (2011, methodology): “In its general form, the residual term is similar to those obtained in PAC-Bayes setting [46, 41, 56] in that it is proportional to the Kullback-Leibler divergence between two probability distributions.”
Tighter PAC-Bayes Bounds (2006, background): “The tightest bounds for practical applications appear to be the PAC-Bayes bound [4, 5].”
An Information-Theoretic View for Deep Learning (2018, background): “ only focus some of the aspects. For example, the VC dimension (Vapnik 2013), covering number (Zhang 2002), Rademacher (Bartlett and Mendelson 2002, Bartlett et al. 2005, Liu et al. 2017), PAC-Bayes (Langford and Shawe-Taylor 2003), algorithmic stability (Liu et al. 2017, Bousquet and Elissee˛ 2002)”
PAC-Bayesian Theory for Transductive Learning (2014, background): “Any upper-bound for the Gibbs classifier’s risk RS′(GQ) can straightforwardly be turned into a bound of the majority vote classifier’s risk RS′(BQ), as RS′(BQ) ≤ 2RS′(GQ) [Langford and Shawe-Taylor, 2002].”
Exact minimax risk for linear least squares, and the lower tail of sample covariance matrices (2019, methodology): “Instead, as in [53], we use the so-called PAC-Bayesian inequality for empirical processes [47, 48, 40, 16, 5], which is based on a variational representation of the relative entropy.”
SPECTRALLY-NORMALIZED MARGIN BOUNDS FOR NEURAL NETWORKS (2018, methodology): “The lemma is analogous to similar analysis of Langford & Shawe-Taylor (2003) and McAllester (2003) obtaining PAC-Bayes margin bounds for linear predictors, and the proof, presented in Section 4, is essentially the same.”
Improvability Through Semi-Supervised Learning: A Survey of Theoretical Results (2019, background): “The starting point is the observation that bounds for randomized classiﬁers are generally tighter when compared to their deterministic counterparts (McAllester, 2003a; Langford and Shawe-Taylor, 2002).”
Tutorial on Practical Prediction Theory for Classification (2005, methodology): “Applying the PAC-Bayes bound requires specialization (Langford and Shawe-Taylor, 2002).”
Implicit Regularization in Deep Learning (2017, methodology): “The proof of the lemma uses similar ideas as in the proof for the case of linear separators, discussed by Langford and Shawe-Taylor [36] and McAllester [35].”
PAC-Bayesian learning of linear classifiers (2009, methodology): “Building on the previous work of Langford and Shawe-Taylor (2003) and Langford (2005), we choose both the prior Pwp and the posterior Qw to be spherical Gaussians with identity covariance matrix respectively centered on wp and on w.”
PAC-Bayesian Inequalities for Martingales (2011, methodology): “PAC-Bayesian analysis supplied generalization guarantees for many influential machine learning algorithms, including support vector machines [14], [15], linear classifiers [16], and clustering-based models [17], to name just a few of them.”
Uniform convergence may be unable to explain generalization in deep learning (2019, methodology): “[23] John Langford and John Shawe-Taylor.”
Learning Efficient Random Maximum A-Posteriori Predictors with Non-Decomposable Loss Functions (2013, methodology): “Interestingly, when incorporating perturbmax models to Bayesian loss minimization one would ultimately like to use the PAC-Bayesian risk [11, 19, 3, 20, 5, 10].”