Discriminative Least Squares Regression for Multiclass Classification and Feature Selection

By Shiming Xiang, Feiping Nie, Gaofeng Meng, Chunhong Pan, Changshui Zhang. IEEE Transactions on Neural Networks and Learning Systems, 2012

432 citations recorded by the source corpus.

This paper presents a framework of discriminative least squares regression (LSR) for multiclass classification and feature selection. The core idea is to enlarge the distance between different classes under the conceptual framework of LSR. First, a technique called ε-dragging is introduced to force the regression targets of different classes moving along opposite directions such that the distances between classes can be enlarged. Then, the ε-draggings are integrated into the LSR model for multiclass classification. Our learning framework, referred to as discriminative LSR, has a compact model form, where there is no need to train two-class machines that are independent of each other. With its compact form, this model can be naturally extended for feature selection. This goal is achieved in terms of L2,1 norm of matrix, generating a sparse learning model for feature selection. The model for multiclass classification and its extension for feature selection are finally solved elegantly and efficiently. Experimental evaluation over a range of benchmark datasets indicates the validity of our method.

Subfields: Artificial Intelligence, Computational Mechanics, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/187110ea2bb38456b73fc50399bd7ce5f622702a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

Feature Selection Based on Structured Sparsity: A Comprehensive Study (2017, methodology): “2) Six SSFS methods, including the L1 [24], [25], [40], DLSR-FS [88], RFS [26], CRFS [86], FS20 [104], and UDFS [98].”
Learning Consistent Feature Representation for Cross-Modal Multimedia Retrieval (2015, background): “We call it as the -dragging [46], which should be learned from both of the modality features.”
Ensemble manifold regularized sparse low-rank approximation for multiview feature embedding (2015, neutral): “In this paper, we focus on the `2,1-norm fitting constraint since it is robust against to noise in the data [30, 31, 32]:”
Robust Sparse Linear Discriminant Analysis (2019, methodology): “integrated this constraint into the linear regression framework to adaptively select those important features for classification [36].”
A Unified Feature Selection Framework for Graph Embedding on High Dimensional Data (2015, methodology): “Finally, embedded methods encapsulate the feature selection problem into their optimization objectives such as the solution sparsity [25], [26], [27], [28].”
Supervised, Unsupervised, and Semi-Supervised Feature Selection: A Review on Gene Selection (2016, methodology): “, 2012) [54] Discriminative least square regression (DLSR) & Multi-tissue [95]; KNN (1-NN) k-fold CV (k 1⁄4 10); Complexity analysis; Paired Students’”