High-Performance Kernel Machines With Implicit Distributed Optimization and Randomization

By Haim Avron, Vikas Sindhwani. 2014

24 citations recorded by the source corpus.

We propose a framework for massive-scale training of kernel-based statistical models, based on combining distributed convex optimization with randomization techniques. Our approach is based on a block-splitting variant of the alternating directions method of multipliers, carefully reconfigured to handle very large random feature matrices under memory constraints, while exploiting hybrid parallelism typically found in modern clusters of multicore machines. Our high-performance implementation supports a variety of statistical learning tasks by enabling several loss functions, regularization schemes, kernels, and layers of randomized approximations for both dense and sparse datasets, in an extensible framework. We evaluate our implementation on large-scale model construction tasks and provide a comparison against existing sequential and parallel libraries. Supplementary materials for this article are available online.

Subfields: Artificial Intelligence, Computational Mechanics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

Quasi-Monte Carlo Feature Maps for Shift-Invariant Kernels (2014, methodology): “This technique is immensely successful, and has been used in recent years to obtain state-of-the-art accuracies for some classical datasets [Huang et al., 2014, Sindhwani and Avron, 2014].”
Local Group Invariant Representations via Orbit Embeddings (2016, background): “The proposed approach can be suitable for largescale problems, benefiting from the recent advances in scalability of randomized kernel methods [3, 13, 25].”