An Introduction to Variable and Feature Selection

By Isabelle M Guyon, Andre´ Elisseeﬀ. 2003

16,826 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Evaluation & Benchmarks

Citations in the corpus

Kernel Mean Embedding of Distributions: A Review and Beyonds (2016, background): “For instance, Song et al. (2007b) considers a supervised feature selection as maximizing a dependence between the subsets of covariate X ′, obtained from backward elimination algorithm (Guyon and Elisseeff 2003), and the target Y , e.g., outputs of binary, multi-class, and regression settings.”
Adversarial Feature Selection Against Evasion Attacks (2016, methodology): “Feature selection is an important preprocessing step in pattern recognition [38], [44], [45].”
Interpreting linear support vector machine models with heat map molecule coloring (2011, background): “A reason for this low probability might be that several patterns that by themselves cannot separate the two classes well might separate the classes better if combined [18].”
A survey on measuring indirect discrimination in machine learning (2015, background): “There are many established measures in feature selection literature [Guyon and Elisseeff 2003] for measuring the relation between two variables, which, in principle, can be used as absolute discrimination measures.”
Benchmark and Survey of Automated Machine Learning Frameworks (2019, methodology): “A simple domain-agnostic filtering approach for feature selection is based on information theory and statistics (Pudil et al., 1994; Yang and Pedersen, 1997; Dash and Liu, 1997; Guyon and Elisseeff, 2003).”
Survey on Automated Machine Learning (2019, background): “By examining the features from an information theoretic or statistics viewpoint (Pudil et al., 1994; Yang and Pedersen, 1997; Dash and Liu, 1997; Guyon and Elisseeff, 2003), no domain knowledge is necessary for feature selection.”
Mathematical optimization in classification and regression trees (2021, background): “This can be done after the model has been built, through variable importance measures, deleting those variables with a small importance (Cohen et al. 2007; Guyon and Elisseeff 2003).”
Supersparse linear integer models for optimized medical scoring systems (2015, methodology): “Sparse linear classifiers can also be produced using feature selection algorithms (Guyon and Elisseeff, 2003, Mao, 2004), though these algorithms cannot guarantee an optimal balance between accuracy and sparsity as they typically rely on greedy optimization (with some exceptions, see e.g. Bradley…”
Supersparse Linear Integer Models for Predictive Scoring Systems (2013, methodology): “These methods can produce very sparse models through heavy regularization or feature selection methods (Guyon and Elisseeff 2003); however, feature selection often relies on greedy optimization and cannot guarantee an optimal balance between sparsity and accuracy.”
How to Explain Individual Classification Decisions (2009, methodology): “Typically, explanations are provided jointly for all instances of the training set, for example feature selection methods (including Automatic Relevance Determination) ﬁnd out which inputs are salient for a good generalization (see for a review Guyon and Elisseeﬀ, 2003).”
Evaluating the Visualization of What a Deep Neural Network Has Learned (2015, background): “Note that this aspect differs from feature selection [12], where the question is: which features are on average salient for the ensemble of training data? Only recently, the transparency problem has been receiving more attention for general nonlinear estimators [13]–[16].”
A Survey on Multi-view Learning (2013, neutral): “This last type of algorithm bears some connections with the mature feature selection algorithms (Jain and Zongker, 1997; Guyon and Elisseeff, 2003); however, there are significant differences between multiview feature selection and single-view feature selection.”
Methods and Models for Interpretable Linear Classification (2014, background): “The interpretability of transparent models is usually improved by tuning sparsity [107, 120, 29, 50, 89, 19, 59, 47], by ensuring monotonic relationships between certain input variables and the predicted outcome [10, 87, 114, 77], and by restricting coefficients to a small set of integer values [23,”
A data-driven approach to predict the success of bank telemarketing (2014, methodology): “In the second step, an automated selection approach is adopted, based an adapted forward selection method [12].”
Computational Phenotype Discovery Using Unsupervised Feature Learning over Noisy, Sparse, and Irregular Clinical Data (2013, background): “But they only find features with task-specific utility, they are as a rule not as effective as features engineered by an expert for a specific task [20], and they don’t seek to provide a complete representation of the input data.”
Failure diagnosis using decision trees (2004, background): “There has been much related work in the area of feature selection [1] [12], and specifically in the context of decision trees [10].”
Kernel Methods for Deep Learning (2009, methodology): “More specifically, in the first step, we discretize each real-valued feature and construct class-conditional and marginal histograms of its discretized values; then, using these histograms, we estimate each feature’s mutual information with the class label and sort the features in order of these est”
Debugging Machine Learning Tasks (2016, background): “There is a lot of work in feature selection [3], and parameter choices can be made by systematically building models for various parameter values and choosing the model with the best validation score [4].”
Touchalytics: On the Applicability of Touchscreen Input as a Behavioral Biometric for Continuous Authentication (2012, methodology): “We use a Gaussian radial-basis function as the kernel, parameterized by the width parameter γ .”
Modeling wine preferences by data mining from physicochemical properties (2009, background): “…António Cerdeira b, Fernando Almeida b, Telmo Matos b, José Reis a,b a Department of Information Systems/R&D Centre Algoritmi, University of Minho, 4800-058 Guimarães, Portugal b Viticulture Commission of the Vinho Verde Region (CVRVV), 4050-501 Porto, Portugal a b s t r a c ta r t i c l e i n f o”