On Bias, Variance, 0/1—Loss, and the Curse-of-Dimensionality

By Jerome H. Friedman. 2004

1,112 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Survey on Automated Machine Learning (2019, background): “This phenomenon is known as curse of dimensionality (Friedman, 1997).”
Sum-product networks: A new deep architecture (2011, background): “…include: (a) it potentially reduces computational cost by allowing max nodes to be merged; (b) ignoring priors may improve discriminative performance [8]; (c) priors may be approximately encoded by the number of units representing the same pattern, and this may facilitate online hard EM learning.”
Review of "Mining the Web: discovering knowledge from hypertext data" by Soumen Chakrabati. Morgan Kaufman 2003. (2002, neutral): “For such discrete regression settings, it is possible to have highly biased and yet effective classifiers if their variance is low, as explained by Friedman, Domingos, and Pazzani [82, 70].”
A multiscale multi-permeability poroplasticity model linked by recursive homogenizations and deep learning (2018, background): “This so-called curse of high dimensionality (cf. Friedman [1997]) is further complicated by the new set of information afforded by recent X-ray tomographic imaging and digital image correlation techniques.”
Classifier ensembles: Select real-world applications (2008, background): “However, Näıve Bayes classifiers or k-nearest neighbor (kNN) classifiers are stable, which explains why bagging is not particularly effective for Näıve Bayes or kNN classifiers [39].”
Survey on Feature Selection (2015, methodology): “For an accurate classiﬁer, it is needed to reduce both bias and variance of the model (Friedman, 1997).”
Tree Induction vs. Logistic Regression: A Learning-Curve Analysis (2003, background): “Bagging reduces variance, which leads to estimation errors as well as classification errors (Friedman, 1997). The ability of Laplace correction and bagging to improve probability estimation of induced trees has been noted previously. Bauer and Kohavi (1999) show similar results evaluating probabilit”
A Uni(cid:12)ed Bias-Variance Decomposition (background): “Friedman (1997) 6 Future Work The main limitation of the de(cid:12)nitions of bias and variance proposed here is that many loss functions cannot be decomposed according to them and Equation 1 (e.g., absolute loss).”
A Practical Approach to Microarray Data Analysis (2003, background): “, bias and variance) of classifiers when assessing or comparing the performance of different classifiers (Friedman, 1996; Breiman, 1998).”
GraphMAE: Self-Supervised Masked Graph Autoencoders (2022, background): “However, MSE is known to suffer from varied feature vector norms and the curse of dimensionality [5] and thus can cause the collapse of training for autoencoders.”
Facial expression recognition from video sequences: temporal and static modeling (2003, background): “However, in the presence of authorities, the Japanese viewers were more reluctant to show their real expressions.”
Activity Recognition in the Home Using Simple and Ubiquitous Sensors (2004, background): “One theory on why naive Bayes classifiers work so well is that the low variance of the classifier can offset the effect of the high bias that results from the strong independence assumptions [6].”
Active Feature-Value Acquisition (2009, methodology): “Friedman’s analysis of the relationship between training data and classification error (Friedman 1997) examines how changes in an induction technique’s average estimation of the probability of the true class affect the likelihood of classification error.”