Reconstruction-based Unsupervised Feature Selection: An Embedded Approach

By Jundong Li, Jiliang Tang, Huan Liu. 2017

64 citations recorded by the source corpus.

Feature selection has been proven to be effective and efficient in preparing high-dimensional data for data mining and machine learning problems. Since real-world data is usually unlabeled, unsupervised feature selection has received increasing attention in recent years. Without label information, unsupervised feature selection needs alternative criteria to define feature relevance. Recently, data reconstruction error emerged as a new criterion for unsupervised feature selection, which defines feature relevance as the capability of features to approximate original data via a reconstruction function. Most existing algorithms in this family assume predefined, linear reconstruction functions. However, the reconstruction function should be data dependent and may not always be linear especially when the original data is high-dimensional. In this paper, we investigate how to learn the reconstruction function from the data automatically for unsupervised feature selection, and propose a novel reconstruction-based unsupervised feature selection framework REFS, which embeds the reconstruction function learning process into feature selection. Experiments on various types of real-world datasets demonstrate the effectiveness of the proposed framework REFS.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Nojun Kwak
2. Seonguk Park
3. Jiliang Tang
4. Huan Liu
5. Jundong Li
6. Suhang Wang
7. Fred Morstatter
8. Kewei Cheng

In Computer Vision and Pattern Recognition

1. Nojun Kwak
2. Seonguk Park
3. Jiliang Tang
4. Jundong Li
5. Fred Morstatter
6. Huan Liu
7. Suhang Wang
8. Kewei Cheng

In Molecular Biology

1. Nojun Kwak
2. Seonguk Park
3. Huan Liu
4. Jiliang Tang
5. Jundong Li
6. Fred Morstatter
7. Suhang Wang
8. Kewei Cheng

Citations in the corpus

• FEED: Feature-level Ensemble for Knowledge Distillation (2019, methodology): “Reconstruction errors were used as a criterion for feature selection or PCAs in (Boutsidis, Mahoney, and Drineas 2008; Farahat, Ghodsi, and Kamel 2011; Li, Tang, and Liu 2017; Masaeli et al. 2010), where they use linear models.”
• Feature-Level Ensemble Knowledge Distillation for Aggregating Knowledge from Multiple Networks (2020, methodology): “Reconstruction errors were used as a criterion for feature selection or PCAs in [2, 5, 16, 17], where they use linear models.”
• Feature Selection (2016, methodology): “…Nguyen et al. 2014; Shishkin et al. 2016; Gao et al. 2016), feature ability to preserve data manifold structure (He et al. 2005; Zhao and Liu 2007; Gu et al. 2011b; Jiang and Ren 2011), and feature ability to reconstruct the original data (Masaeli et al. 2010; Farahat et al. 2011; Li et al. 2017a).”