Generalized Fisher Score for Feature Selection

By Quanquan Gu, Zhenhui Jessie Li, Jiawei Han. Conference on Uncertainty in Artificial Intelligence, 2011

848 citations recorded by the source corpus.

Fisher score is one of the most widely used supervised feature selection methods. However, it selects each feature independently according to their scores under the Fisher criterion, which leads to a suboptimal subset of features. In this paper, we present a generalized Fisher score to jointly select features. It aims at finding an subset of features, which maximize the lower bound of traditional Fisher score. The resulting feature selection problem is a mixed integer programming, which can be reformulated as a quadratically constrained linear programming (QCLP). It is solved by cutting plane algorithm, in each iteration of which a multiple kernel learning problem is solved alternatively by multivariate ridge regression and projected gradient descent. Experiments on benchmark data sets indicate that the proposed method outperforms Fisher score as well as many other state-of-the-art feature selection methods.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/464c737376f34c5d38d92940f711abd94d37e84f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Exploring discriminative pose sub-patterns for effective action classification (2013, methodology): “As a SSP actually corresponds to a feature subset, it can be evaluated with Fisher score based feature selection method [17].”
Feature Selection (2016, methodology): “2016], feature ability to preserve data manifold structure [He et al. 2005; Zhao and Liu 2007; Gu et al. 2011b; Jiang and Ren 2011], and feature ability to reconstruct the original data [Masaeli et al.”
A Survey of Modern Questions and Challenges in Feature Extraction (2015, methodology): “Filter methods select a subset of features based on the notion of an importance score that is computed independent of the classifier (Gu et al., 2012; Boutsidis et al., 2009; He et al., 2005; Chen and Lin, 2006).”
Classification and Prediction of Brain Disorders Using Functional Connectivity: Promising but Challenging (2018, methodology): “Fisher score is a univariate feature selection algorithm which has been applied to determine the discriminatory power of features between two groups with equal probability (Gu et al., 2012; Khazaee et al., 2015).”
A Unified Feature Selection Framework for Graph Embedding on High Dimensional Data (2015, methodology): “Since the number of constraints in P is no longer large, this problem is readily solved by a sub-gradient method, such as simpleMKL [18], [42].”
DLow: Diversifying Latent Flows for Diverse Human Motion Prediction (2020, methodology): “Similarly, Fisher information [60] has been used for diverse feature [23] and data [64] selection.”
Feature Selection for Classification: A Review (2014, background): “[17] proposed a generalized Fisher score to jointly select features, which aims to find an subset of features that maximize the lower bound of traditional Fisher score and solve the following problem:”
Feature Selection Based on Structured Sparsity: A Comprehensive Study (2017, methodology): “1) Nine traditional feature selection methods, including chisquare (CS), variance, Fisher score (FS) [12], [13], Gini coefficient (Gini) [159], information gain (IG) [160], minimum redundancy maximum relevance (mRMR) [161], relief [162], students T-test (T-test) [163], Kruskal–Wallis test (K-test) [”