Efficient Spectral Feature Selection with Minimum Redundancy

By Zheng Zhao, Lei Wang, Huan Liu. AAAI Conference on Artificial Intelligence, 2010

296 citations recorded by the source corpus.

Spectral feature selection identifies relevant features by measuring their capability of preserving sample similarity. It provides a powerful framework for both supervised and unsupervised feature selection, and has been proven to be effective in many real-world applications. One common drawback associated with most existing spectral feature selection algorithms is that they evaluate features individually and cannot identify redundant features. Since redundant features can have significant adverse effect on learning performance, it is necessary to address this limitation for spectral feature selection. To this end, we propose a novel spectral feature selection algorithm to handle feature redundancy, adopting an embedded model. The algorithm is derived from a formulation based on a sparse multi-output regression with a L2,1-norm constraint. We conduct theoretical analysis on the properties of its optimal solutions, paving the way for designing an efficient path-following solver. Extensive experiments show that the proposed algorithm can do well in both selecting relevant features and removing redundancy.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Signal Processing

View Paper: https://www.semanticscholar.org/paper/7155733b3562fda3f29689c3d4cbfe7da2aa94d0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Unsupervised Feature Selection with Structured Graph Optimization (2016, methodology): “2007), wrapper method (Tabakhi, Moradi, and Akhlaghian 2014), and embedded method (Cai, Zhang, and He 2010; Zhao, Wang, and Liu 2010; Hou et al. 2014; Li et al. 2012; Wang, Tang, and Liu 2015).”
Generalized Fisher Score for Feature Selection (2011, result): “This is consistent with the observation in [24].”
High-Dimensional Feature Selection by Feature-Wise Kernelized Lasso (2012, background): “Table 4 shows the RED values for the top m = 50 features selected by each method.”
Pattern Recognition (2018, unknown): “Considering the above problems, we summarize our contributions as follows: • We study gait recognition and soft biometrics in a joint framework with convolutional neural networks (CNNs).”
Federated Multi-view Learning for Private Medical Data Integration and Analysis (2021, methodology): “• Single View w/o FL [56]: In this approach, we optimize the following objective function on each view: where Y in (20) reasons is that the special view provides less information than the other two views.”
Feature Selection Based on Structured Sparsity: A Comprehensive Study (2017, background): “, multicluster feature selection [92] and minimum redundancy spectral feature selection [93] can be regarded as special cases of JELSR.”
Robust Joint Graph Sparse Coding for Unsupervised Spectral Feature Selection (2017, methodology): “The reason is that: 1) the 1-norm regularizer outputs the sparsity in elements, thus prohibiting for feature selection on multiclass data and 2) the 2,1-norm regularizer outputs the sparsity through the whole row (corresponding to one feature of the data) and has been widely used in real application”
Feature Selection (2016, background): “Many `2,1-norm regularization based feature selection methods have been proposed over the past decade [Zhao et al. 2010; Gu et al. 2011c; Yang et al. 2011; Hou et al. 2011; Li et al. 2012; Qian and Zhai 2013; Shi et al. 2014; Liu et al. 2014; Du and Shen 2015; Jian et al. 2016; Liu et al. 2016b; Nie”
Joint Embedding Learning and Sparse Regression: A Framework for Unsupervised Feature Selection (2014, methodology): “See more details in [30].”
A review of unsupervised feature selection methods (2019, methodology): “For this evaluation, the redundancy rate (Zheng et al. 2010) and Representation Entropy (Devijver and Kittler 1982) are the most used redundancy measures.”