On Similarity Preserving Feature Selection

By Zheng Zhao, Lei Wang, Huan Liu, Jieping Ye. IEEE Transactions on Knowledge and Data Engineering, 2013

316 citations recorded by the source corpus.

In the literature of feature selection, different criteria have been proposed to evaluate the goodness of features. In our investigation, we notice that a number of existing selection criteria implicitly select features that preserve sample similarity, and can be unified under a common framework. We further point out that any feature selection criteria covered by this framework cannot handle redundant features, a common drawback of these criteria. Motivated by these observations, we propose a new "Similarity Preserving Feature Selection” framework in an explicit and rigorous way. We show, through theoretical analysis, that the proposed framework not only encompasses many widely used feature selection criteria, but also naturally overcomes their common weakness in handling feature redundancy. In developing this new framework, we begin with a conventional combinatorial optimization formulation for similarity preserving feature selection, then extend it with a sparse multiple-output regression formulation to improve its efficiency and effectiveness. A set of three algorithms are devised to efficiently solve the proposed formulations, each of which has its own advantages in terms of computational complexity and selection performance. As exhibited by our extensive experimental study, the proposed framework achieves superior feature selection performance and attractive properties.

Subfields: Computer Vision and Pattern Recognition, Management Science and Operations Research

View Paper: https://www.semanticscholar.org/paper/49800883dd0dbc9462ba5a4d25b12c6bbf26475e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Dacheng Tao
2. Luc Van Gool
3. Lei Zhang
4. Ming-Hsuan Yang
5. Yi Yang
6. Xiaogang Wang
7. Wangmeng Zuo
8. Chunhua Shen

In Management Science and Operations Research

1. Dacheng Tao
2. Tien-Ping Tan
3. Jie Gui
4. Qinghua Hu
5. Anant Madabhushi
6. Wangmeng Zuo
7. Shuiwang Ji
8. Zhenan Sun

Citations in the corpus

• Unsupervised text feature selection technique based on hybrid particle swarm optimization algorithm with genetic operators for the text clustering (2017, unknown): “Generally, cosine similarity is the common measurement that is used in the text mining area and particularly in the text clustering domain to compute the similarity value between each document with the clusters centroids [30].”
• Unsupervised feature selection by regularized self-representation (2015, background): “Feature selection methods can be categorized into unsupervised and supervised ones [4,5,13].”
• Clustering-Guided Sparse Structural Learning for Unsupervised Feature Selection (2014, methodology): “1) Baseline: All original features are adopted; 2) MaxVar: Features corresponding to the maximum variance are selected to obtain the best expressive features; 3) LS [21]: Features consistent with Gaussian Laplacian matrix are selected to best preserve the local manifold structure [21]; 4) SPEC [7]: ”
• Feature Importance in Nonlinear Embeddings (FINE): Applications in Digital Pathology (2016, methodology): “sequential feature selection) in conjunction with NLDR to preserve the structure of the embedding while eliminating redundant features [34], [35], [36].”
• Feature Selection Based on Structured Sparsity: A Comprehensive Study (2017, methodology): “The differences of SPFS and RFS are detailed in [83].”
• Graph Learning for Multiview Clustering (2018, unknown): “Besides with Gaussian function, there are several graph construction methods, such as local linear similarity graph [1], k-nearest neighbor graph [2], [3], local discriminant graph [4], [5], pairwise similarity graph [6], and graph learned with subspace clustering [7].”
• Feature selection for text classification: A review (2018, methodology): “…features [103], online feature selection [99], filter-based locality preserving feature selection [36], similarity preserving feature selection [112], feature selection with optimization techniques [105], regularization based feature selection methods [121], feature selection with machine…”