Temporal Evolution of Generalization during Learning in Linear Networks

By Pierre Baldi, Yves Chauvin. 1991

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Scaling & Learning Theory

Citations in the corpus

High-dimensional dynamics of generalization error in neural networks (2017, background): “This is related to a large body of prior work on shallow networks (Chauvin, 1990; P. Baldi and Chauvin, 1991; LeCun, Kanter, and Solla, 1991; Seung, Sompolinsky, and Tishby, 1992; Dodier, 1995; P.F. Baldi and Hornik, 1995; Kinouchi and Caticha, 1995; Dunmur and Wallace, 1999; Krogh and Hertz, 1992;…”
Minnorm training: an algorithm for training over-parameterized deep neural networks (2018, background): “However, the loss surface for this network is non-convex [3, 12, 24] and the solution manifold for this simple case is the hyperbola w 1 w 2 = 1 .”
Geometry of Early Stopping in Linear Networks (1995, unknown): “Omitting a few steps of algebra,E(w) = 12 pXk=1 kv2k + 12y0(y XwLS) (1)where v = S0(w wLS) and = diag( 1; : : : ; p).”
Automatic early stopping using cross validation: quantifying the criteria (1998, background): “(Baldi & Chauvin, 1991) showed for linear networks with n inputs and n outputs that up to n such local minima are possible; for multi layer networks, the situation is even worse.”
Early Stopping-But When? (1996, result): “4 Why this works Detailed theoretical analyses of the error curves cannot yet be done for the most interesting cases such as sigmoidal multi-layer perceptrons trained on a modest number of examples; today they are possible for restricted scenarios only [1, 2, 3, 24] and do usually not aim at nding t”
A New Adaptive Merging and Growing Algorithm for Designing Artificial Neural Networks (2009, background): “It has been shown for linear networks with Ψ inputs and Ψ outputs that up to Ψ local minima are possible; for multilayer ANNs, the situation is even worse [63].”