Four Types of Learning Curves

By Shun-ichi Amari, Naotake Fujita, Shigeru Shinomoto. 1992

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Evaluation & Benchmarks

Citations in the corpus

Deep Learning Scaling is Predictable, Empirically (2017, methodology): “Using statistical mechanics approaches, Amari et al. (1992) and Amari (1993) show that as sample complexity grows, generalization error should decline as a power-law ε ( m ) ∼ αm β g with β g = − 0 .”
Scaling Laws for Neural Machine Translation (2021, background): “power(x_d , p[2]) + p[3] - y def fit_model(x, y, f_scale): X = x.”
The Dipping Phenomenon (2012, background): “Such behavior can indeed be demonstrated in certain settings in which the classifier selected typically fits the underlying data assumptions well, see for instance [1,10].”
A Least-squares Approach to Direct Importance Estimation (2009, background): “(7) Below, we theoretically investigate the learning curve (Amari et al., 1992) of LSIF, that is, we elucidate the relation between J(α̂(λ)) and J(α∗(λ)) in terms of the expectation over all possible training and test samples as a function of the number of samples.”
Rigorous Learning Curve Bounds from Statistical Mechanics (2005, methodology): “Our formalism can be used to give a classification of the largeα asymptotics of scaled learning curves 7, thus completing a classification program that has been suggested by several researchers (Amari et al., 1992; Schwartz et al., 1990; Seung et al., 1992).”
A Scaling Law for Syn2real Transfer: How Much Is Your Pre-training Effective? (2021, background): “Neural scaling laws The scaling behavior of generalization error, including some theoretical works [e.g., 3], has been studied extensively.”
Statistical Theory of Learning Curves under Entropic Loss Criterion (1993, background): “Therefore, the present paper complements the result by Amari et al. (1992) where learning curves are obtained for deterministic machines under the annealed approximation.”