Feature Selection with Ensembles, Artificial Variables, and Redundancy Elimination

By Eugene Tuv, Alexander Borisov, George C. Runger, Kari Torkkola. 2009

346 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Benchmark and Survey of Automated Machine Learning Frameworks (2019, methodology): “Many ML models provide some sort of feature ranking that can be utilized, e.g., SVMs (Guyon et al., 2002; Rakotomamonjy, 2003), perceptrons (Mej́ıa-Lavalle et al., 2006) or random forests (Tuv et al., 2009).”
Gradient boosted feature selection (2014, neutral): “Learning”
Understanding Random Forests: From Theory to Practice (2014, methodology): “At the price of more computational efforts, even better performance can usually be reached by embedding variable im- 162 further insights from importances portances into a dedicated iterative feature selection procedure, such as those described in [Guyon et al., 2002; Tuv et al., 2009].”
On Oblique Random Forests (2011, background): “More recently, additional properties of the random forest have gained interest, for example in feature selection [25, 34, 23, 44] or the explorative analysis of sample proximities [38].”
Analysis of the AutoML Challenge Series 2015-2018 (2019, methodology): “No pre-processing was done, except for feature selection (Tuv et al., 2009).”
Feature Selection: A literature Review (2014, methodology): “ Combining with other techniques such as tree ensemble [86], and feature extraction [87]”
Robustness of Random Forest-based gene selection methods (2013, methodology): “RF-ACE performs a predefined number of iterations (the default value used in this study is 20) and, at each step, collects the importance of real features and the mean importance of all shadows.”
Once Upon a Crime: Towards Crime Prediction from Demographics and Mobile Data (2014, neutral): “) [41].”
Please Stop Permuting Features: An Explanation and Alternatives (2019, background): “Similar ideas also appear in Tuv et al. (2009) and also (more specific to the context of linear regression) in Wu et al. (2007).”
Improving Gender Classification of Blog Authors (2010, methodology): “Finally, we are aware that there are some existing ensemble feature selection methods in the machine learning literature ( Garganté et al., 2007; Tuv et al., 2009 ).”
Random Forest ensembles for detection and prediction of Alzheimer's disease with a good between-cohort robustness (2014, background): “Although RF can itself be considered as an effective feature selection algorithm, several approaches for feature set reduction within and outside the context of RF have been proposed to further improve its performance(Tuv et al., 2009).”
A review of feature selection methods on synthetic data (2013, methodology): “Therefore, new feature selection methods are constantly appearing using different strategies: a) combining several feature selection methods, which could be done by using algorithms from the same approach, such as two filters [16], or coordinating algorithms from two different approaches, usually fi”
Feature selection based on artificial bee colony and gradient boosting decision tree (2019, methodology): “We combine xgboost [30] with 10-fold cross-validation to evaluate the accuracy of classification with feature selections. x i ← v ij , trial i = 1 13: else if rand ( 0 , 1 ) < p t 19: x i = v ij , trial i = 1 ifmax( trial i ) > lim it 28: x ij = x min , j + rand ( 0 , F m ( x ) = 0,i=1,N for m =1…”
A brief Review of the ChaLearn AutoML Challenge: Any-time Any-dataset Learning without Human Intervention (2016, methodology): “No pre-processing was done, except for feature selection (Tuv et al., 2009).”