The statistics of critical points of Gaussian fields on large-dimensional spaces

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Entropy-SGD: biasing gradient descent into wide valleys (2016, background): “They also relate to earlier studies such as Baldi & Hornik (1989); Fyodorov & Williams (2007); Bray & Dean (2007) which show that critical points with high training error are exponentially likely to be saddle points with many negative directions and all local minima are likely to have error that is…”
An empirical analysis of the optimization of deep network loss surfaces (2016, background): “Some have characterized the critical points of deep linear neural networks [2, 28] and high-dimensional, random Gaussian error functions [4, 9, 11, 24].”
The Loss Surfaces of Multilayer Networks (2014, background): “Earlier theoretical analyses [Baldi and Hornik, 1989, Wigner, 1958, Fyodorov and Williams, 2007, Bray and Dean, 2007] suggest the existence of a certain structure of critical points of random Gaussian error functions on high dimensional continuous spaces.”
Deep relaxation: partial differential equations for optimizing deep neural networks (2017, background): “Based on empirical results such as Dauphin et al. (2014), authors in Bray and Dean (2007); Fyodorov and Williams (2007); Choromanska et al. (2015) and Chaudhari and Soatto (2015) have modeled the energy landscape of deep neural networks as a high-dimensional Gaussian random ﬁeld.”