Semi-Proximal Mirror-Prox for Nonsmooth Composite Minimization

By Niao He, Zaïd Harchaoui. 2015

15 citations recorded by the source corpus.

We propose a new first-order optimization algorithm to solve high-dimensional non-smooth composite minimization problems. Typical examples of such problems have an objective that decomposes into a non-smooth empirical risk part and a non-smooth regularization penalty. The proposed algorithm, called Semi-Proximal Mirror-Prox, leverages the saddle point representation of one part of the objective while handling the other part of the objective via linear minimization over the domain. The algorithm stands in contrast with more classical proximal gradient algorithms with smoothing, which require the computation of proximal operators at each iteration and can therefore be impractical for high-dimensional problems. We establish the theoretical convergence rate of Semi-Proximal Mirror-Prox, which exhibits the optimal complexity bounds, i.e. O(1/∊2), for the number of calls to linear minimization oracle. We present promising experimental results showing the interest of the approach in comparison to competing methods.

Subfields: Artificial Intelligence, Computational Mechanics, Numerical Analysis, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Mechanics

In Numerical Analysis

In Optimization & Training

Citations in the corpus

Frank-Wolfe Algorithms for Saddle Point Problems (2016, methodology): “Finally, He and Harchaoui (2015) introduced an interesting extragradient-type method to solve (3) by approximating the projections using linear oracles.”