Joint Embedding Learning and Sparse Regression: A Framework for Unsupervised Feature Selection

By Chenping Hou, Feiping Nie, Xuelong Li, Dong-yun Yi, Yi Wu. 2014

533 citations recorded by the source corpus.

Feature selection has aroused considerable research interests during the last few decades. Traditional learning-based feature selection methods separate embedding learning and feature ranking. In this paper, we propose a novel unsupervised feature selection framework, termed as the joint embedding learning and sparse regression (JELSR), in which the embedding learning and sparse regression are jointly performed. Specifically, the proposed JELSR joins embedding learning with sparse regression to perform feature selection. To show the effectiveness of the proposed framework, we also provide a method using the weight via local linear approximation and adding the ℓ2,1-norm regularization, and design an effective algorithm to solve the corresponding optimization problem. Furthermore, we also conduct some insightful discussion on the proposed feature selection approach, including the convergence analysis, computational complexity, and parameter determination. In all, the proposed framework not only provides a new perspective to view traditional methods but also evokes some other deep researches for feature selection. Compared with traditional unsupervised feature selection methods, our approach could integrate the merits of embedding learning and sparse regression. Promising experimental results on different kinds of data sets, including image, voice data and biological data, have validated the effectiveness of our proposed algorithm.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Urban Studies, Representation & Self-Supervised Learning, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Urban Studies

In Representation & Self-Supervised Learning

In Optimization & Training

Citations in the corpus

Predicting multicellular function through multi-layer tissue networks (2017, methodology): “The data matrix is projected to a low dimensional space using linear techniques based on SVD (Tang et al. , 2012), or non-linear techniques based on multi-dimensional scaling (Tenenbaum et al. , 2000; Belkin and Niyogi, 2001; Hou et al. , 2014).”
A Survey of Sparse Representation: Algorithms and Applications (2015, background): “More detailed analysis on matching pursuit algorithms can be found in the literature [63].”
On Better Exploring and Exploiting Task Relationships in Multitask Learning: Joint Model and Feature Learning (2018, methodology): “Existing algorithms mainly use two methods to measure the relatedness between tasks: shared common models/parameters [5, 6, 7, 8, 9, 10] and shared common feature representations [11, 12, 13, 14, 15, 16, 17].”
A Unified Feature Selection Framework for Graph Embedding on High Dimensional Data (2015, methodology): “Finally, embedded methods encapsulate the feature selection problem into their optimization objectives such as the solution sparsity [25], [26], [27], [28].”
Unsupervised Feature Selection with Structured Graph Optimization (2016, methodology): “And more than 6.4% improvement for handwritten digit data sets (i.e. BA, USPS, UMIST), compared to RSFS, which is the second best approach in handwritten digit data sets.”
Feature Selection Based on Structured Sparsity: A Comprehensive Study (2017, methodology): “4) Feature Selection via Joint Embedding Learning and Sparse Regression: Instead of regressing each example to its label [26], [86], [88], the objective of joint embedding learning and sparse regression (JELSR) [89], [90] is to regress each example Xi to its low-dimensional embedding Zi ∈ Rm , where”