On Bayesian bounds

By Arindam Banerjee. 2006

93 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Mechanics, Management Science and Operations Research, Probabilistic & Bayesian Methods, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Mechanics

In Management Science and Operations Research

In Probabilistic & Bayesian Methods

In Statistical Learning Theory

Citations in the corpus

PAC-Bayesian Theory Meets Bayesian Inference (2016, neutral): “To bridge the Bayesian approach with the PAC-Bayesian framework, we consider the negative log-likelihood loss function [3], denoted `nll and defined by `nll(fθ, x, y) ≡ − ln p(y|x, θ) .”
Tighter Variational Representations of f-Divergences via Restriction to Probability Measures (2012, background): “…theorem specialises to show that for any choice of φ ∈ L ∞ ( Q ) The ﬁrst inequality is the well-known representation of the KL divergence in the large deviations literature (Donsker & Varadhan, 1983) which has been rediscovered in the PAC-Bayes community as the compression lemma (Banerjee, 2006).”
User-friendly introduction to PAC-Bayes bounds (2021, background): “Note that these connections are well-known and were discussed in the literature, see for example [20].”
Learning to share visual appearance for multiclass object detection (2011, methodology): “, Banerjee (2007). In particular, if p , q for some constant 0 < q < 1, then R Bayes(Z,θ ∗) is at most ‖θ∗‖2 2σ2 +m ln n 1− q + ln 1 q + m 2 ln ( 1 + Tcσ(2) m ) .”
Covariance in Unsupervised Learning of Probabilistic Grammars (2010, methodology): “This decoding method is attractive because it has generalization error guarantees: in a PAC-Bayesian framework, it can be shown that the error of committee parsing on the sample given should be close to the expected error (see Seeger, 2002; McAllester, 2003; Banerjee, 2006).”
PAC-Bayesian learning of linear classifiers (2009, background): “See Banerjee (2006) for a very similar proof.”
PAC-Bayesian Bounds based on the Rényi Divergence (2016, methodology): “Note that the same result is derived from Fenchel’s inequality [Banerjee, 2006] and Donsker-Varadhan’s variational formula for relative entropy [Seldin et al., 2012, Tolstikhin and Seldin, 2013].”
Risk bounds for the majority vote: from a PAC-Bayesian analysis to a learning algorithm (2015, methodology): “Note that the same result is derived from Fenchel’s inequality in Banerjee (2006) and Donsker-Varadhan’s variational formula for relative entropy in Seldin et al.”
PAC-Bayesian Analysis of Co-clustering and Beyond (2010, methodology): “Banerjee (2006) terms it acompression lemma; however we find the term “change of measure inequality” more appropriate to its nature and usage.”
Disentangling Label Distribution for Long-tailed Visual Recognition (2020, methodology): “We utilize the optimal form of the regularized DonskerVaradhan (DV) representation [15, 3, 4, 12] to model the log-likelihood ratio above explicitly.”