A Reduction of the Elastic Net to Support Vector Machines with an Application to GPU Computing
By Quan Zhou, Shiji Song, Jacob V. Gardner, Kilian Q. Weinberger, Yixin Chen. 2014
46 citations recorded by the source corpus.
Algorithmic reductions are one of the corner stones of theoretical computer science. Surprisingly, to-date, they have only played a limited role in machine learning. In this paper we introduce a formal and practical reduction between two of the most widely used machine learning algorithms: from the Elastic Net (and the Lasso as a special case) to the Support Vector Machine. First, we derive the reduction and summarize it in only 11 lines of MATLAB. Then, we demonstrate its high impact potential by translating recent advances in parallelizing SVM solvers directly to the Elastic Net. The resulting algorithm is a parallel solver for the Elastic Net (and Lasso) that naturally utilizes GPU and multi-core CPUs. We evaluate it on twelve real world data sets, and show that it yields identical results as the popular (and highly optimized) glmnet implementation but is up-to two orders of magnitude faster.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Vanya Van Belle
- Lieven Billiet
- S. Van Huffel
- Yixin Chen
- Zhicheng Cui
- Weiwei Jiang
- Cynthia Rudin
- Berk Ustun
In Computer Vision and Pattern Recognition
- Yixin Chen
- S. Van Huffel
- Yujie He
- Vanya Van Belle
- Lieven Billiet
- Zhicheng Cui
- Aynaz Taheri
- Christopher Morris
Citations in the corpus
- Interval Coded Scoring: a toolbox for interpretable scoring systems (2018, background): “Zhou et al. (2015) have shown that the elastic net is equivalent to the dual formulation of a squared-hinge loss support vector machine.”
- Optimal Action Extraction for Random Forests and Boosted Trees (2015, background): “This is akin to the spirit of other reduction methods in machine learning [32, 10] which reduce a new problem to another well-established problem to exploit its extensive research and development.”
- Interval Coded Scoring extensions for larger problems (2017, methodology): “[14] show that that the constrained quadratic problem expressed in (5) is equivalent to a dual SVM notation, giving rise to the Support Vector Elastic Net method (SVEN).”