Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity

By Hédy Attouch, Zaki Chbani, Juan Peypouquet, Patrick Redont. Mathematical programming, 2018

364 citations recorded by the source corpus.

In a Hilbert space setting $${{\mathcal {H}}}$$H, we study the fast convergence properties as $$t \rightarrow + \infty $$t→+∞ of the trajectories of the second-order differential equation $$\begin{aligned} \ddot{x}(t) + \frac{\alpha }{t} \dot{x}(t) + \nabla \Phi (x(t)) = g(t), \end{aligned}$$x¨(t)+αtx˙(t)+∇Φ(x(t))=g(t),where $$\nabla \Phi $$∇Φ is the gradient of a convex continuously differentiable function $$\Phi : {{\mathcal {H}}} \rightarrow {{\mathbb {R}}}, \alpha $$Φ:H→R,α is a positive parameter, and $$g: [t_0, + \infty [ \rightarrow {{\mathcal {H}}}$$g:[t0,+∞[→H is a small perturbation term. In this inertial system, the viscous damping coefficient $$\frac{\alpha }{t}$$αt vanishes asymptotically, but not too rapidly. For $$\alpha \ge 3$$α≥3, and $$\int _{t_0}^{+\infty } t \Vert g(t)\Vert dt < + \infty $$∫t0+∞t‖g(t)‖dt<+∞, just assuming that $${{\mathrm{argmin\,}}}\Phi \ne \emptyset $$argminΦ≠∅, we show that any trajectory of the above system satisfies the fast convergence property $$\begin{aligned} \Phi (x(t))- \min _{{{\mathcal {H}}}}\Phi \le \frac{C}{t^2}. \end{aligned}$$Φ(x(t))-minHΦ≤Ct2.Moreover, for $$\alpha > 3$$α>3, any trajectory converges weakly to a minimizer of $$\Phi $$Φ. The strong convergence is established in various practical situations. These results complement the $${{\mathcal {O}}}(t^{-2})$$O(t-2) rate of convergence for the values obtained by Su, Boyd and Candès in the unperturbed case $$g=0$$g=0. Time discretization of this system, and some of its variants, provides new fast converging algorithms, expanding the field of rapid methods for structured convex minimization introduced by Nesterov, and further developed by Beck and Teboulle with FISTA. This study also complements recent advances due to Chambolle and Dossal.

Subfields: Computational Theory and Mathematics, Mathematical Physics

View Paper: https://www.semanticscholar.org/paper/5b9d3f1da11694c18601ba1c0fac0f16e8b39f95

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Mathematical Physics

Citations in the corpus

Fast optimization via inertial dynamics with closed-loop damping (2020, background): “Based on the dynamic approach above, many recent studies have been devoted to the convergence properties of the associated algorithms, which have led to a better understanding and improvement of Nesterov’s accelerated gradient algorithm; see [11], [19], [22], [23], [25], [32], [54], [81].”