Early Stopping-But When?

By Lutz Prechelt. 1996

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

Citations in the corpus

Don't Decay the Learning Rate, Increase the Batch Size (2017, background): “Supporting this view, we note that most researchers employ early stopping (Prechelt, 1998), whereby we intentionally prevent the network from reaching a minimum.”
Gradient-Based Neural DAG Learning (2019, background): “This approach is known as early stopping (Prechelt, 1997).”
Recent advances in convolutional neural networks (2015, background): “For example, the training might be performed for a fixed number of epochs, or until a predefined training error is reached [141].”
Early Stopping without a Validation Set (2017, methodology): “Finally, a widely-used provision against overfitting is to add regularization terms to the objective function that penalize the parameter vector w, typically measured by the l1 or l2 norm [9].”
A jamming transition from under- to over-parametrization affects generalization in deep learning (2018, background): “Some of them focus on the eﬀects of various ways of regularizing the network, thereby eﬀectively reducing the dimension [21, 22, 23, 24].”
Inference Compilation and Universal Probabilistic Programming (2016, methodology): “Minibatches of this “infinite” stream of training data are discarded after each SGD update; we therefore have no notion of a finite training set and associated issues such as overfitting to a set of training data and early stopping using a validation set (Prechelt, 1998).”
A comprehensive survey on machine learning for networking: evolution, applications and research opportunities (2018, methodology): “They leverage the Early Stopping Validation Method [366] to reduce training and validation time of the NN to less than 5 hours.”
Self-ensembling for visual domain adaptation (2017, methodology): “We found that using the proportion of samples that passed the conﬁdence threshold can be used to drive early stopping ([19]).”
A Transdisciplinary Review of Deep Learning Research and Its Relevance for Water Resources Scientists (2017, unknown): “Early stopping criteria are often based on a separate validation data set and some smoothness criteria; for example, it stops when the error on the validation data set starts to increase compared to earlier epochs (Prechelt, 2012).”
How to Generate a Good Word Embedding (2015, methodology): “The most widely used early stopping method is to stop the iterator when the loss on the validation set peaks [27].”
Mortality assessment in intensive care units via adverse events using artificial neural networks (2006, background): “Finally, the training is stopped when the error slope is approaching zero or after a maximum of 100 epochs [22].”
A Deep CNN-LSTM Model for Particulate Matter (PM2.5) Forecasting in Smart Cities (2018, background): “However, there are many ways to avoid overﬁtting, such as regularization [28], data augmentation [22], dropout [29], dropconnect [30], or early stopping [31]. dropout [29], dropconnect [30], or early stopping [31].”
BERT Loses Patience: Fast and Robust Inference with Early Exit (2020, methodology): “This is the well-known overﬁtting problem which can be resolved by applying an early stopping mechanism [7, 8].”
Deep packet: a novel approach for encrypted traffic classification using deep learning (2017, methodology): “Additionally, to avoid the over-fitting problem, we have used early stopping technique [33].”
Fast relational learning using bottom clause propositionalization with artificial neural networks (2013, unknown): “We evaluated six different configurations of CILP++ in order to explore some of the capabilities of the approach: three versions of CILP++ trained with standard backpropagation, each one using three sizes of background knowledge, and three versions of CILP++ trained with early stopping (Prechelt 199”