Efficient and robust feature extraction by maximum margin criterion

By Jun Liu, Songcan Chen, Xiaoyang Tan, Daoqiang Zhang. 2003

970 citations recorded by the source corpus.

In pattern recognition, feature extraction techniques are widely employed to reduce the dimensionality of data and to enhance the discriminatory information. Principal component analysis (PCA) and linear discriminant analysis (LDA) are the two most popular linear dimensionality reduction methods. However, PCA is not very effective for the extraction of the most discriminant features, and LDA is not stable due to the small sample size problem . In this paper, we propose some new (linear and nonlinear) feature extractors based on maximum margin criterion (MMC). Geometrically, feature extractors based on MMC maximize the (average) margin between classes after dimensionality reduction. It is shown that MMC can represent class separability better than PCA. As a connection to LDA, we may also derive LDA from MMC by incorporating some constraints. By using some other constraints, we establish a new linear feature extractor that does not suffer from the small sample size problem, which is known to cause serious stability problems for LDA. The kernelized (nonlinear) counterpart of this linear feature extractor is also established in the paper. Our extensive experiments demonstrate that the new feature extractors are effective, stable, and efficient.

Subfields: Analytical Chemistry, Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Analytical Chemistry

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Towards Better Analysis of Deep Convolutional Neural Networks (2016, methodology): “In CNNVis, we provide several options to define the importance of a neuron, including its average or maximal activation on a set of classes and its contribution to the final result [37].”
Human–Agent Teaming for Multirobot Control: A Review of Human Factors Issues (2014, neutral): “proposed the maximum margin criterion method [43].”
Maximum Margin Criterion (2018, methodology): “[20] and it can outperform PCA and Linear Discriminant Analysis (LDA) on”
Sparse Representation Based Fisher Discrimination Dictionary Learning for Image Classification (2014, methodology): “, minimize tr(SW (X)) − a • tr(SB(X)), where a is a positive constant to balance the contributions of within-class scatter and between-class scatter (Li et al. 2006; Song et al. 2007; Guo et al. 2003; Wang et al. 2007).”
A Unified Framework for Metric Transfer Learning (2017, methodology): “using the Euclidean distance may be suboptimal: commonly they would not maximize the inter-class distances while minimizing intra-class distances [18], [19], [20], [21].”
Very low resolution face recognition problem (2010, positive): “Therefore, inspired by maximum margin criterion (MMC) [17], we design a discriminative constraint as follows: d(R) = mean({‖Ii h − RI l ‖2|Ω(Ii h) = Ω(I l )}) − mean({‖Ii h − RI l ‖2|Ω(Ii h) = Ω(I l )}) (13)”
Non-I.I.D. Multi-Instance Dimensionality Reduction by Learning a Maximum Bag Margin Subspace (2010, background): “Maximum Margin Criterion (MMC) (Li and Jiang 2003) and some variants (Yan et al.”
Feature Selection Based on Structured Sparsity: A Comprehensive Study (2017, methodology): “Inspired by MMC, a unsupervised maximum margin feature selection algorithm via sparse constraints is proposed in [97].”
From Manifold to Manifold: Geometry-Aware Dimensionality Reduction for SPD Matrices (2014, methodology): “which resembles the Maximum Margin Criterion (MMC) of [11].”
Maximum Neighborhood Margin Discriminant Projection for Classification (2014, background): “Based on the idea of the maximum margin criterion [43], (19) can be reformulated as follows: or According to (20) or (21), we can find two aspects that are favorable for classification.”
Linear discriminant analysis for the small sample size problem: an overview (2014, background): “A number of papers have been reported for the cancer classification task using the microarray data (Dudoit et al., 2002; Li et al., 2003, 2005; Moghaddam et al., 2006; Sharma and Paliwal, 2008a).”
Motor Bearing Fault Diagnosis Using Trace Ratio Linear Discriminant Analysis (2014, methodology): “D visualization and classification performance of the data demonstrate that the extended TR-LDA method achieves better performance than other algorithms, such as PCA, LPP, MMC, CCA LDA, and MFA, in fault diagnosis.”
Manifold Discriminant Analysis (2009, methodology): “To improve LDA, many extensions have been developed, such as Nonparametric Discriminant Analysis (NDA) [5], Subclass Discriminant Analysis (SDA) [33], Maximum Margin Criterion (MMC) [18], and Kernel Fisher Discriminant (KFD) [19].”