PAC-Bayes-Empirical-Bernstein Inequality

By Ilya O. Tolstikhin, Yevgeny Seldin. 2013

110 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Scaling & Learning Theory

Citations in the corpus

PAC-Bayesian Theory Meets Bayesian Inference (2016, background): “However, despite the amount of work dedicated to this statistical learning theory—many authors improved the initial results [9, 29, 33, 43, 48] and/or generalized them for various machine learning setups [4, 14, 20, 28, 38, 44, 45, 46]—it is mostly used as a frequentist method.”
Risk bounds for the majority vote: from a PAC-Bayesian analysis to a learning algorithm (2015, methodology): “Note that the same result is derived from Fenchel’s inequality in Banerjee (2006) and Donsker-Varadhan’s variational formula for relative entropy in Seldin et al. (2012); Tolstikhin and Seldin (2013).”
Bayesian Counterfactual Risk Minimization (2018, methodology): “To achieve a similar rate, we could perhaps use a PACBayesian Bernstein inequality [29, 34].”
User-friendly introduction to PAC-Bayes bounds (2021, methodology): “Plugging this in Seeger’s inequality leads to Tolstikhin and Seldin’s bound [189].”
PAC-Bayesian Bounds based on the Rényi Divergence (2016, unknown): “Note that the same result is derived from Fenchel’s inequality [Banerjee, 2006] and Donsker-Varadhan’s variational formula for relative entropy [Seldin et al., 2012, Tolstikhin and Seldin, 2013].”
Meta-Learning by Adjusting Priors Based on Extended PAC-Bayes Theory (2017, unknown): “Finally we note that more recent works presented possibly tighter PAC-Bayes bounds by taking into account the empirical variance (Tolstikhin & Seldin, 2013), or by specializing the bound to deep neural networks (Neyshabur et al., 2018) or by using more general divergences than the KL divergence…”