Theory II: Landscape of the Empirical Risk in Deep Learning

By Tomaso A. Poggio, Qianli Liao. 2017

81 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

The Loss Surface of Deep and Wide Neural Networks (2017, background): “On the theoretical side there have been several interesting developments recently, see e.g. (Brutzkus & Globerson, 2017; Lee et al., 2016; Poggio & Liao, 2017; Rister & Rubin, 2017; Soudry & Hoffer, 2017; Zhou & Feng, 2017).”
Theory of Deep Learning III: explaining the non-overfitting puzzle (2017, background): “• unlike shallow networks deep networks can approximate the class of hierarchically local functions without incurring in the curse of dimensionality (2, 22); • overparametrized deep networks yield many global degenerate – and thus flat – or almost degenerate minima (3) which are selected by SGD with”
A Closer Look at Deep Learning Heuristics: Learning rate restarts, Warmup and Distillation (2018, background): “Many of these ingredients are now commonplace in modern architectures, and some of them have also been buttressed with theoretical guarantees (Balduzzi et al., 2017; Poggio & Liao, 2017; Hardt & Ma, 2016).”
Generating Accurate Pseudo-Labels in Semi-Supervised Learning and Avoiding Overconfident Predictions via Hermite Polynomial Activations (2019, unknown): “For example, [22] studied deep ReLU networks by using a polynomial non-linearity as an approximation for ReLU: this enabled a much cleaner analysis of the empirical risk landscape.”
Neural network models and deep learning - a primer for biologists (2019, background): “Beyond the motivation just given, the stochasticity is thought to contribute also to finding solutions that generalize well beyond the training set (Poggio and Liao, 2017).”
Nonparametric regression using deep neural networks with ReLU activation function (2017, background): “For more on the energy landscape, see [32].”
Designing a Moral Compass for the Future of Computer Vision Using Speculative Analysis (2017, unknown): “Again, this is distinct from other deﬁnitions of risk, such as empirical risk embedded in certain methodological choices [47].”