Convergence of the Iterates of Descent Methods for Analytic Cost Functions

By Pierre-Antoine Absil, Robert E. Mahony, Ben Andrews. SIAM Journal on Optimization, 2005

376 citations recorded by the source corpus.

Subfields: Computational Theory and Mathematics, Numerical Analysis

View Paper: https://www.semanticscholar.org/paper/e6d88754cb0f64fc1a4728bb13ffe3863fd771ad

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Numerical Analysis

Citations in the corpus

Clarke Subgradients of Stratifiable Functions (2006, methodology): “An example of a similar nature has been given in [1] and will be described below: Let f be defined in polar coordinate on R(2) by”
Convergence Analysis of a Momentum Algorithm with Adaptive Step Size for Non Convex Optimization (2019, methodology): “This framework was used for gradient descent (Absil et al., 2005), for proximal gradient descent (see Attouch and Bolte (2009) for an analysis with the Lojasiewicz inequality) and further generalized to a class of descent methods called gradient-like descent algorithms.”
Gradient Descent Converges to Minimizers (2016, background): “The first part of the theorem follows from [1], which shows tha t limk xk exists.”
Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods (2013, methodology): “This type of quadratic models arises, for instance, in trust-region methods (see [1] which is also connected to Lojasiewicz inequality).”
Algorithmic Regularization in Learning Deep Homogeneous Models: Layers are Automatically Balanced (2018, background): “Then from (14) we have Under the above descent condition, the result of Absil et al. [2005] says that the iterates either diverge to inﬁnity or converge to a ﬁxed point.”
CONVERGENCE TO EQUILIBRIUM FOR THE BACKWARD EULER SCHEME AND APPLICATIONS (2010, methodology): “Notice that a related question has been answered for some explicit descent methods by Absil, Mahony and Andrews in a context of optimization [1].”
The [barred L]ojasiewicz Inequality for Nonsmooth Subanalytic Functions with Applications to Subgradient Dynamical Systems (2006, background): “For related results on this topic, see [1] ; for other applications to partial differential equations, see the works of Simon [21] and Haraux [12].”
Backtracking gradient descent method for general $C^1$ functions, with applications to Deep Learning (2018, background): “The Mexican hat example in [1] is as follows (see Equation 2.8 therein).”
On the convergence of the proximal algorithm for nonsmooth functions involving analytic features (2008, methodology): “fail even for a finite-valued smooth function f , see Palis and De Melo [25 ]o r Absil et al. [ 1 ].”
On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transport (2018, background): “Similarly, when there is compactness, a gradient flow that does not converge is an unexpected (in some sense adversarial) behavior, see a counter-example in [2].”
Characterizations of Lojasiewicz inequalities: Subgradient flows, talweg, convexity (2009, methodology): “This condition has been considered in [1] for nonconvex functions defined in finitedimensional spaces.”
Gradient Descent Maximizes the Margin of Homogeneous Neural Networks (2019, methodology): “It is known that gradient flow (or gradient descent) may not converge to any point even when optimizing an C∞ function (Curry, 1944; Zoutendijk, 1976; Palis and De Melo, 2012; Absil et al., 2005).”
Gradient Descent Only Converges to Minimizers (2016, background): “Proof The first part of the theorem follows from Absil et al. (2005), which shows that lim k x k exists.”
From error bounds to the complexity of first-order descent methods for convex functions (2015, methodology): “eﬁnition is due to the fact that such sequences are generated by many prominent methods, such as the forward-backward method [50, 4, 35] (which we describe in detail below), many trust region methods [1], alternating methods [4, 21], and, in a much more subtle manner, sequential quadratic methods an”
Characterizations of Lojasiewicz inequalities and applications (2008, methodology): “This condition has been considered in [1] for nonconvex functions defined in finite-dimensional spaces.”
On Convergence of Model Parallel Proximal Gradient Algorithm for Stale Synchronous Parallel System (2016, methodology): “It has recently been used to analyze discrete-time algorithms in [1] and proximal algorithms in [3, 4, 11].”
Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Lojasiewicz Inequality (2008, result): “Similar results have been developed for discrete gradient methods (see [1]) and nonsmooth subanalytic functions (see [16, 17, 4]).”