The Problem of Overfitting

By Douglas M. Hawkins. 2004

2,546 citations recorded by the source corpus.

Subfields: Analytical Chemistry, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Analytical Chemistry

In Statistics and Probability

Citations in the corpus

Evolutionary algorithms for hyperparameter tuning on neural networks models (2014, methodology): “Fitness Operator: In order to avoid overfitting (Hawkins 2004) we have used as fitness function the validation error of the ANN over the dataset.”
Automatically Designing CNN Architectures Using Genetic Algorithm for Image Classification (2018, background): “However, the fullyconnected layer easily results into the over-fitting phenomenon [40] due to its dense connection [41].”
Conceptual and empirical comparison of dimensionality reduction algorithms (PCA, KPCA, LDA, MDS, SVD, LLE, ISOMAP, LE, ICA, t-SNE) (2021, background): “, The curse of dimensionality in data mining and time series prediction, (International WorkConference on Artificial Neural Networks (2005), Springer), 758-770 [7] Hawkins, D.”
A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects (2020, background): “After convolution, feature maps consist of a large number of features that is prone to causing overfitting problem [21].”
Application of Mid-infrared and Raman Spectroscopy to the Study of Bacteria (2011, background): “predictive coefficients into a model may mean that random variation is incorporated into future predictions (Hawkins 2004).”
Automatically Designing CNN Architectures Using the Genetic Algorithm for Image Classification (2018, background): “However, the fully connected layer easily results in the overfitting phenomenon [40] due to its dense connection [41].”
Reliable estimation of prediction errors for QSAR models under model uncertainty using double cross-validation (2014, neutral): “This well documented phenomenon is also called overfitting and is frequently addressed in the literature [2,24-28].”
Importance of sample size, data type and prediction method for remote sensing-based estimations of aboveground forest biomass (2014, background): “The reason is that going towards more complex models increases the risk of overfitting, i.e. explaining variance through a large number of redundant variables, which may lead to undesirably high variance on parameter estimates and unreliable predictions (see e.g. Hawkins, 2004).”
Review of Image Classification Algorithms Based on Convolutional Neural Networks (2021, background): “This mechanism makes the operation simple and efficient, and can operate on a very large-scale dataset, which greatly reduces the amount of training parameters and reduces the risk of overfitting [65].”
Reproducible brain-wide association studies require thousands of individuals (2022, neutral): “6.”
Brain-Machine Interfaces: From Basic Science to Neuroprostheses and Neurorehabilitation. (2017, background): “known in many disciplines as overfitting (38, 338, 345) or the curse of dimensionality (821).”
Deep Learning for Household Load Forecasting—A Novel Pooling Deep RNN (2018, neutral): “data diversity and network parameter redundancy [39].”
The Persistence and Transience of Memory. (2017, background): “Overﬁtting refers to a pernicious problem in statistics and machine learning: when overly precise models are ﬁt to a ﬁnite dataset, it leads to inaccurate predictions due to the focus on the particulars of that dataset (Hawkins, 2004).”
Regularizing Deep Convolutional Neural Networks with a Structured Decorrelation Constraint (2016, background): “Deep networks are usually composed of multiple layers containing a tremendous amount of trainable parameters, leading to some serious problems like vanishing gradient in back propagation [9], diminishing feature reuse in forward propagation [10] and overfitting [11, 12].”
On further application of r m2 as a metric for validation of QSAR models (2009, methodology): “[20,32,57] already showed that for small sample size, LOO cross-validation with application of model selection process at each cycle of iterations during validation performs better than the traditional splitting process for validation.”