Entropy-SGD optimizes the prior of a PAC-Bayes bound: Generalization properties of Entropy-SGD and data-dependent priors

67 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Sharpness-Aware Minimization for Efficiently Improving Generalization (2020, methodology): “The connection between the geometry of the loss landscape—in particular, the flatness of minima— and generalization has been studied extensively from both theoretical and empirical perspectives (Shirish Keskar et al., 2016; Dziugaite & Roy, 2017; Jiang et al., 2019).”
Wireless Network Intelligence at the Edge (2018, methodology): “• PAC-Bayes Bound for Entropy SGD A recent work [131] verifies that Entropy SGD works by optimizing the Bayesian prior, i.”
Data-dependent Sample Complexity of Deep Neural Networks via Lipschitz Augmentation (2019, background): “Dziugaite & Roy (2017) optimize a bound to compute non-vacuous bounds for generalization error.”
Non-vacuous Generalization Bounds at the ImageNet Scale: a PAC-Bayesian Compression Approach (2018, background): “Other authors have explored modiﬁcations of the training procedure to obtain networks with provable generalization guarantees (Dziugaite & Roy, 2017; 2018).”