Automatic early stopping using cross validation: quantifying the criteria

By Lutz Prechelt. 1998

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Style transfer-based image synthesis as an efficient regularization technique in deep learning (2019, methodology): “stop the optimization procedure (early stopping [5]), to select the initial model weights (transfer learning [6]).”
fPINNs: Fractional Physics-Informed Neural Networks (2018, background): “se measurements for both forwardandinverseproblems. Additionally,thenoiseinthedatacanbenaturallytakeninto accountbyemployingregularizationtechniques,suchasL 1,L2 andL regularization[32], earlystopping[33],aswellasdropout[34,35]. (2) They can easily handle high-dimensional, irregular-domain problems.”
3D radiotherapy dose prediction on head and neck cancer patients with a hierarchically densely connected U-net deep learning architecture (2018, methodology): “The validation loss is used to determine the iteration that had the best model weights, a well-known method to help prevent from obtaining a model that has overfitted to the data (Prechelt 1998, Caruana et al 2001).”
A New Adaptive Merging and Growing Algorithm for Designing Artificial Neural Networks (2009, background): “In [71], a number of plausible criteria are proposed for terminating the”
Addressing two problems in deep knowledge tracing via prediction-consistent regularization (2018, methodology): “e possible extensions of this work is to take the further future interaction into account when training the DKT model: L= 1 c 0 BBB BBB @ Xn i=1 TX i1 t=1 TX t j=1 j1l yi t (q t+j);a 1 CCC CCC A (13) where c = P n i=1 P T i 1 t =1 P i t j j 1 is the normalizing term, is the decay factor similar to”
Regularization for Deep Learning: A Taxonomy (2017, methodology): “∙ Termination using a validation set – Early stopping (see Morgan and Bourlard, 1990; Prechelt, 1998) – Choice of validation set size based on test set size (Amari et al.”
Comparison between M5 Model Tree and Neural Networks for Estimating Reference Evapotranspiration in an Arid Environment (2014, methodology): “The most common technique to circumvent overfitting is based on an early stopping criterion that halts training before convergence (Sarle 1995; Prechelt 1998).”
Machine learning exciton dynamics (2015, unknown): “Target feature over-fitting was avoided by using early stopping [62].”
Nonlinear mode decomposition with convolutional neural networks for fluid dynamics (2019, methodology): “For training both CNNs, we apply the early stopping criterion (Prechelt 1998) to avoid overfitting and use the Adam algorithm (Kingma & Ba 2014) to seek the optimized weights w.”
Ensure the Correctness of the Summary: Incorporate Entailment Knowledge into Abstractive Sentence Summarization (2018, methodology): “Our model with entailmentaware encode requires less than 300,000 training iterations to train with early stopping (Prechelt, 1998).”
Neural-Network-Based Models for Short-Term Traffic Flow Forecasting Using a Hybrid Exponential Smoothing and Levenberg–Marquardt Algorithm (2012, methodology): “4) Cross-validation based LM algorithm [LM-CROSS-(τ)] uses the mechanisms of cross-validation [2], [37] to avoid overtraining NNs.”
Learning Deep Representation for Face Alignment with Auxiliary Attributes (2014, methodology): “Neural network methods [25] have also extensively used it to prevent over-fitting by halting the training process of a single task before a minimum error is achieved on the training set.”
Deep Learning over Multi-field Categorical Data - - A Case Study on User Response Prediction (2016, methodology): “une hyperparameters in the models. We use stochastic gradient descent to learn most of our parameters for all proposed models. Regarding selecting the number of training epochs, we use early stopping [30], i.e., the training stops when the validation error increases. We try diﬀerent learning rate fr”
Facial Landmark Detection by Deep Multi-task Learning (2014, methodology): “Neural network methods [20] have also extensively used it to prevent over-fitting by halting the training process of a single task before a minimum error is achieved on the training set.”