Parallel Selective Algorithms for Nonconvex Big Data Optimization

By Francisco Facchinei, Gesualdo Scutari, Simone Sagratella. 2014

172 citations recorded by the source corpus.

We propose a decomposition framework for the parallel optimization of the sum of a differentiable (possibly nonconvex) function and a (block) separable nonsmooth, convex one. The latter term is usually employed to enforce structure in the solution, typically sparsity. Our framework is very flexible and includes both fully parallel Jacobi schemes and Gauss-Seidel (i.e., sequential) ones, as well as virtually all possibilities “in between” with only a subset of variables updated at each iteration. Our theoretical convergence results improve on existing ones, and numerical results on LASSO, logistic regression, and some nonconvex quadratic problems show that the new method consistently outperforms existing algorithms.

Subfields: Artificial Intelligence, Computational Mechanics, Numerical Analysis

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Mechanics

In Numerical Analysis

Citations in the corpus

DJAM: Distributed Jacobi Asynchronous Method for Learning Personal Models (2018, methodology): “The methods proposed in [15] and [16] can tackle more general problems, but both require that agents communicate with all their neighbors before updating, while our method needs only communications between two agents at a time.”
On Nonconvex Decentralized Gradient Descent (2016, background): “Some typical applications include: (i) the distributed compressed sensing problems [14], [30], [39], [45], [49]; (ii) distributed consensus [9], [29], [55], [58], [61], [69]; (iii) distributed and parallel machine learning [15], [21], [33], [43], [55].”
Joint Optimization of Radio and Computational Resources for Multicell Mobile-Edge Computing (2014, methodology): “Some relevant instances of this method that have attracted significant interest in recent years are: i) the basic DCA (Difference-of-Convex Algorithm) [34], [35]; ii) the M(ajorization)-M(inimization) algorithm [36], [37]; iii) alternating/successive minimization methods [38]–[40]; and iv) partial l”
A Framework for Parallel and Distributed Training of Neural Networks (2016, methodology): “The algorithm, which leverages on the so-called successive convex approximation (SCA) family of methods (Facchinei et al., 2015), is built upon two foundational ideas.”
Majorization-Minimization Algorithms in Signal Processing, Communications, and Machine Learning (2017, background): “To arrive at a stationary point, FLEXA [107]–[109] approximates f by a strongly convex function g (·|xt) satisfying the property that ∇g (xt |xt) = ∇f (xt).”