Fisher discriminant analysis with kernels

By Sebastian Mika, Gunnar Rätsch, Jason Aaron Edward Weston, Bernhard Schölkopf, K.R. Mullers. 1999

A non-linear classification technique based on Fisher's discriminant is proposed. The main ingredient is the kernel trick which allows the efficient computation of Fisher discriminant in feature space. The linear classification in feature space corresponds to a (powerful) non-linear decision function in input space. Large scale simulations demonstrate the competitiveness of our approach.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Unraveling Meta-Learning: Understanding Feature Representations for Few-Shot Tasks (2020, background): “Linear discriminant analysis (LDA) projects data onto directions that minimize the intra-class to inter-class variance ratio (Mika et al., 1999), and so is ideal for visualizing class separation phenomenon.”
Domain Generalization via Conditional Invariant Representations (2018, methodology): “The scatter component analysis combines domain scatter (Muandet, Balduzzi, and Schölkopf 2013), kernel PCA (Schölkopf, Smola, and Müller 1998), and kernel FDA (Mika et al. 1999) in a single objective function.”
Linear discriminant analysis: A detailed tutorial (2017, unknown): “One solution to this problem is to use the kernel functions as reported in [50].”
Local discriminant embedding and its variants (2005, methodology): “Often, without getting rid of those nice linear properties behind techniques like PCA and LDA, their discriminating power can be enhanced by incorporating nonlinearities via kernel methods [20], [14].”
Kernel Sparse Representation-Based Classifier (2012, methodology): “These methods can directly assign a class label to a test sample without a training process.”
Feature-based transfer learning with real-world applications (2010, methodology): “Similar to kernel Fisher discriminant analysis (KFD) [126, 216], the W solution in (3.”
Person Re-identification: Past, Present and Future (2016, background): “Liao et al. [20] propose learning the projection w to a low-dimensional subspace with cross-view data solved in a similar manner to linear discriminant analysis (LDA) [61], where S b and S w are the between-class and within-class scatter matrices, respectively.”
Supervised Machine Learning: A Review of Classification Techniques (2007, unknown): “When dealing with categorical variables, the equivalent technique is Discriminant Correspondence Analysis (Mika et al., 1999).”
The Geometry Of Kernel Canonical Correlation Analysis (2003, methodology): “Since the geometry of linear CCA is preserved in the kernel variant this relation also holds for the kernel methods (Mika et al. 1999).”
Optimizing the kernel in the empirical feature space (2005, methodology): “Given a training data set , ( ), the different algorithms, such as the support vector machine (SVM) [6], [10], [16], [22], kernel Fisher discriminant (KFD) [4], [15], [20], and kernel principal component analysis (KPCA) [23], perform only in a subspace of the feature space spanned by the images of…”
Efficient and robust feature extraction by maximum margin criterion (2003, background): “Kernel Fisher’s discriminant (KFD) [23] is a well-known nonlinear extension to LDA.”
DehazeNet: An End-to-End System for Single Image Haze Removal (2016, methodology): “For example, PCA [42] and LDA [43] as classical linear lowdimensional mappings are widely used in computer vision and data mining.”
Kernel-based methods for hyperspectral image classification (2005, methodology): “The idea of the KFD analysis is to solve the well-known problem of Fisher's linear discriminant in a kernel feature space, which produces a nonlinear discriminant classifier in the input space [36].”