Asymptotic statistical theory of overtraining and cross-validation

By Shun-ichi Amari, Noboru Murata, K. Müller, Michael Finke, Howard H. Yang. 1997

Subfields: Statistics and Probability, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

In Statistical Learning Theory

Citations in the corpus

Forecasting daily streamflow using hybrid ANN models (2006, background): “However, Amari et al. (1997) showed that overfitting would not occur if the ratio of the number of training data sets to the number of the weights in the network exceeds 30.”
Stochasticity, nonlinearity and forecasting of streamflow processes (2006, background): “Sarle (2002) comments that the results of Amari et al. (1997) contain serious errors that completely invalidate.”
Understanding Machine-learned Density Functionals (2014, background): “[54] S-I Amari, N Murata, K-R Müller, M Finke, and HH Yang, “Asymptotic statistical theory of overtraining and cross-validation,” IEEE Transactions on Neural Networks 8, 985–996 (1997).”
Neural-Network-Based Models for Short-Term Traffic Flow Forecasting Using a Hybrid Exponential Smoothing and Levenberg–Marquardt Algorithm (2012, methodology): “4) Cross-validation based LM algorithm [LM-CROSS-(τ)] uses the mechanisms of cross-validation [2], [37] to avoid overtraining NNs.”
The connection between regularization operators and support vector kernels (1998, methodology): “Besides that model selection principles like structural risk minimization (Vapnik, 1982), cross validation (Bishop, 1995; Amari et al., 1997; Kearns, 1997), MDL (Rissanen, 1985), Bayesian methods (MacKay, 1991; Bishop, 1995), etc. can be employed.”
Neural networks for the prediction and forecasting of water resource variables: a review of modelling issues and applications (2000, methodology): “When the above condition is not met, there are clear beneﬁts in using cross-validation (Amari et al., 1997).”
Early Stopping-But When? (1996, methodology): “See Amari et al. [1, 2 ] for yet another view of the training process, using a geometrical interpretation.”
Adaptive Regularization in Neural Network Modeling (1996, methodology): “Furthermore [2] and [37] consider the use of a validation set to tune the amount of regularization, in particular when using the early-stop technique.”
Artificial neural network simulation of hourly groundwater levels in a coastal aquifer system of the Venice lagoon (2012, background): “Due to the high number of available observations, cross validation on an independent data set has not been exploited to avoid overfitting (Amari et al., 1997).”
Fuzzy model validation using the local statistical approach (2009, background): “to evaluate the adequacy of a fuzzy model (or of the equivalent neural network) and the efficiency of the learning that has been performed is cross-validation [1,24].”