Use of the Zero-Norm with Linear Models and Kernel Methods

By Jason Aaron Edward Weston, Andre´ Elisseeﬀ, Bernhard Schölkopf, Michael E. Tipping. 2003

881 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Adversarial Feature Selection Against Evasion Attacks (2016, neutral): “tures during classifier training [59], [60].”
Mathematical optimization in classification and regression trees (2021, background): “The first enhancement relates to extending orthogonal splits to oblique, a.k.a. multivariate, splits, with implementations such as OC1 (Murthy et al. 1994), oblique. tree (Truong 2009), and HHCART (Wickramarachchi et al. 2016).”
A Biologically Inspired System for Action Recognition (2007, methodology): “.3.4 C 2 feature selection with zero-norm SVM As in [16], we have performed experiments with the zeronorm SVM [31] of Weston et al. Instead of regularizing the norm of the hyperplane ||w|| 2 , the classifier tries to optimize the objective function: ||w|| 0 + C N i=1 ζ i , such that (w T x i + b) > ”
An Introduction to Variable and Feature Selection (2003, methodology): “Several papers in this issue use variable ranking as a baseline method (see, e.g., Bekkerman et al., 2003, Caruana and de Sa, 2003, Forman, 2003, Weston et al., 2003).”
Supervised feature selection via dependence estimation (2007, methodology): “…experiment, we show that the performance of BAHSIC can be comparable to other state-of-the-art feature selectors, namely SVM Recursive Feature Elimination (RFE) (Guyon et al., 2002), RELIEF (Kira & Rendell, 1992), L 0-norm SVM (L 0) (Weston et al., 2003), and R2W2 (Weston et al., 2000).”
Robust Joint Graph Sparse Coding for Unsupervised Spectral Feature Selection (2017, methodology): “[38] added an 0-norm regularizer into the proposed objective function to achieve sparse solution for performing feature selection, while Liu et al.”
Feature selection, L1 vs. L2 regularization, and rotational invariance (2004, background): “Sparse parameter vectors have few non-zero entries Regularization based on the zero-norm maximizes sparseness, but zero-norm minimization is an NP-hard problem (Weston et al. 2003).”
A hybrid genetic algorithm for feature selection wrapper based on mutual information (2007, methodology): “…elimination (RFE) approach, which is a recently proposed feature selection algorithm derived based on the support vector machine (SVM) theory and has shown good performance on the problems of gene selection for the microarray data (Guyon et al., 2002; Weston et al., 2003; Rakotomamonjy, 2003).”
Local Causal and Markov Blanket Induction for Causal Discovery and Feature Selection for Classification Part I: Algorithms and Empirical Evaluation (2010, methodology): “A fourth set uses theL0-norm penalty, for example, feature selection via approximate solution of theL0-norm formulation of SVM classification problem (Weston et al., 2003).”
VSURF: An R Package for Variable Selection Using Random Forests (2015, methodology): “We illustrate the use of the VSURF package on a simple simulated moderately high-dimensional dataset (n = 100 < p = 200) called toys data, introduced by Weston et al. (2003). It comes from an equiprobable two-class problem, Y ∈ {−1, 1}, with 6 influential variables and with the others being noise va”
Fast Optimization Methods for L1 Regularization: A Comparative Study and Two New Approaches (2007, background): “The problem of obtaining an optimal subset of features for a linear classifier is known to be NP-hard [18].”
Kernel Methods and Machine Learning (2014, methodology): “10 (Embedded methods) One of the first embedded methods proposed for feature selection is the 0-norm SVM [298], which optimizes the following objective function:”