Learning by on-line gradient descent

By Michael Biehl, H. Schwarze. 1995

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Continual Learning in the Teacher-Student Setup: Impact of Task Similarity (2021, background): “1a was ﬁrst studied in a series of classic papers by Biehl & Schwarze (1995) and Saad & Solla (1995a) who derived a set of closed ODEs that track the test error of the student (see also Saad & Solla (1995b); Biehl et al. (1996); Saad (2009) for further results and Goldt et al. (2019) for a recent…”
Statistical Mechanics of On-line Learning (2009, background): “With this specific choice, the averaging in the equations of motion (30, 31, 32) can be performed analytically for general K and M [22,23,24] Independent of the particular choice of learning algorithms a general problem in two-layered networks is caused by the inherent permutation symmetry: The i-th”
Dynamics of Learning Near Singularities in Layered Networks (2008, unknown): “…arises from symmetry (see, for example, Riegler & Biehl, 1995; Saad & Solla, 1995a, b; Biehl, Riegler, & Wöhler, 1996; Inoue, Park, & Okada, 2003, 2004; Park, Inoue, & Okada, 2003, 2005; see also Biehl & Schwarze, 1995; Biehl & Caticha, 2002; Freeman & Saad, 1997a, b; Huh, Oh, & Kang, 2000).”
Dynamics of stochastic gradient descent for two-layer neural networks in the teacher–student setup (2019, unknown): “Our main contributions are as follows: (i) The dynamics of SGD (online) learning by two-layer neural networks in the teacher-student setup was studied in a series of classic papers [41, 42, 43, 44, 45] from the statistical physics community, leading to a heuristic derivation of a set of coupled ordi”
Modelling the influence of data structure on learning in neural networks (2019, background): “These “plateaus” are well-known in the literature for both SGD, where they appear as a function of time [35, 22, 36], and in batch learning, where they appear as a function of the training set size [37, 4].”