Accelerated Parallel Optimization Methods for Large Scale Machine Learning
By Haipeng Luo, Patrick Haffner, Jean-François Paiement. 2014
7 citations recorded by the source corpus.
The growing amount of high dimensional data in different machine learning applications requires more efficient and scalable optimization algorithms. In this work, we consider combining two techniques, parallelism and Nesterov's acceleration, to design faster algorithms for L1-regularized loss. We first simplify BOOM, a variant of gradient descent, and study it in a unified framework, which allows us to not only propose a refined measurement of sparsity to improve BOOM, but also show that BOOM is provably slower than FISTA. Moving on to parallel coordinate descent methods, we then propose an efficient accelerated version of Shotgun, improving the convergence rate from $O(1/t)$ to $O(1/t^2)$. Our algorithm enjoys a concise form and analysis compared to previous work, and also allows one to study several connected work in a unified way.
Subfields: Artificial Intelligence, Computational Mechanics, Optimization & Training, Mathematical Optimization
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Computational Mechanics
In Optimization & Training
In Mathematical Optimization
Citations in the corpus
- Parallel Coordinate Descent Newton Method for Efficient $\ell_1$-Regularized Minimization (2013, methodology): “le this problem can be alleviated by preprocessing samples (e.g., feature clustering) to achieve higher parallelism [20], it requires additional computational overhead. The Accelerated Shotgun method [13] is a rst-order algorithm without backtrack line search which has fast convergence. However, it ”