Heuristics of instability and stabilization in model selection

By Leo Breiman. 1996

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Diversity With Cooperation: Ensemble Methods for Few-Shot Classification (2019, methodology): “One way to encourage diversity consists of introducing randomization in the learning procedure, e.g. , by using data augmentation [3, 10] or various initializations.”
Ensemble methods and partial least squares regression (2004, methodology): “This method generates a large number of perturbed data sets by sampling with ∗This is a preprint of an article published in Journal of Chemometrics 2004; 18(11): 498–507, c© 2004 †Corresponding author.”
LASSO-TYPE GMM ESTIMATOR (2009, methodology): “This problem is severe for subset selection methods as shown in linear and some nonlinear cases by Breiman (1996).”
AMlet, RAMlet, and GAMlet: Automatic Nonlinear Fitting of Additive Models, Robust and Generalized, With Wavelets (2004, background): “However, the knot selection procedure is known to suffer from an instability that many practitioners have observed and that Breiman (1996) has studied.”
HIBAG – HLA Genotype Imputation with Attribute Bagging (2013, unknown): “We propose the HIBAG algorithm to impute HLA types, using the bagging method developed by Breiman, 15,20 with improvements of variable subset suggested by Breiman 16 and Bryll et al. , 17 applied to a haplotype-based classiﬁer.”
GENERALIZED DOUBLE PARETO SHRINKAGE. (2011, methodology): “We consider the ozone data analyzed by Breiman and Friedman (1985) and by Casella and Moreno (2006). The original data set contains 13 variables and 366 observations.”
Variable Selection Using Random Forests (2006, methodology): “…instability, a problem occurring when there are many different models with similar predictive accuracy and a slight perturbation in the data or in the model construction can cause a skip from one model to another, close in terms of error, but distant in terms of the meaning (Breiman (1996a)).”
Consistency of Random Survival Forests (2008, unknown): “A widely used ensemble technique is bagging (Breiman, 1996a).”
Variable Selection in Semiparametric Regression Modeling. (2008, unknown): “The Annals of Statistics 2008, Vol. 36, No. 1, 261–286 DOI: 10.1214/009053607000000604 © Institute of Mathematical Statistics, 2008”
Partial linear single index models with distortion measurement errors (2013, background): “However, these variable selection procedures encounter the problem of intensive computation and the lack of stability as argued by Breiman (1996).”
Classification tree analysis using TARGET (2008, background): “Breiman (1996a, 2001b) also discusses the problem of instability in CART solutions, where perturbing a small proportion of the training set or resampling the training set often results in a solution with a very different tree structure.”
Stability and Generalization (2002, methodology): “One such technique is the bagging approach of Breiman (1996a) which consists in averaging several estimators built from random subsamples of the data. Although it is generally accepted that having a low variance (or a high stability in our terminology) is a desirable property for a learning algorith”
One-step Sparse Estimates in Nonconcave Penalized Likelihood Models (2008, unknown): “10 30 v1 [ m at h. ST ] 7 A ug 2 00 8 The Annals of Statistics 2008, Vol. 36, No. 4, 1561–1566 DOI: 10.1214/07-AOS0316REJ Main article DOI: 10.1214/009053607000000802 c© Institute of Mathematical Statistics, 2008”
Decision Tree Models for Prediction of Macroinvertebrate Taxa in the River Axios (Northern Greece) (2007, background): “The instability of a classifier is defined as the tendency to find large changes in the predicted values caused by minor changes in the dataset (Breiman 1996).”
Bolasso: model consistent Lasso estimation through the bootstrap (2008, methodology): “Note in particular that we compare with bagging of least-square regressions (Breiman, 1996a) followed by a thresholding of the loading vector, which is another simple way of using bootstrap samples: the Bolasso provides a more ef.cient way to use the extra information, not for usual…”