A Characterization of Superlinear Convergence and its Application to Quasi-Newton Methods

By John E. Dennis Jr., Jorge J. Moré. 1973

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

Citations in the corpus

Fast and Furious Convergence: Stochastic Second Order Methods under Interpolation (2019, methodology): “Quasi-Newton methods that approximate the Hessian are popular in practice due to their cheaper iteration costs compared to full Newton methods while having superlinear convergence (Dennis and Moré, 1974).”
Optimization Methods for Large-Scale Machine Learning (2016, methodology): “ed along the most recent displacement (though not necessarily along other directions). A remarkable fact about the BFGS iteration (6.8){(6.9) is that it enjoys a local superlinear rate of convergence [48], and this with only rst-order information and without the need for any linear system solves (wh”
Communication-Efficient Distributed Optimization of Self-Concordant Empirical Loss (2015, positive): “In classical analysis of inexact Newton methods [18, 16], asymptotic superlinear convergence occurs with ǫk ∼ ‖f ′(wk)‖ 2 (in fact with ǫ ∼ ‖f (wk)‖2 for any s > 1).”
A descent modified Polak–Ribière–Polyak conjugate gradient method and its global convergence (2006, methodology): “In order to introduce our method, let us simply recall the well-known BFGS quasi-Newton method (see, e.g. Dennis & Moré, 1974, 1977).”
Adaptive cubic regularisation methods for unconstrained optimization. Part II: worst-case function- and derivative-evaluation complexity (2011, background): “3) is a slight strengthening of the Dennis–Moré condition [3].”
Quasi-Newton Methods, Motivation and Theory (1974, unknown): “Another consequence of (8.16) and (8.18) is the local convergence of the DPP and BFGS methods as given by Broyden, Dennis and More (1973).”
Convex Optimization for Big Data: Scalable, randomized, and parallel algorithms for big data analytics (2014, methodology): “I Convergence is guaranteed under the Dennis & Moré condition [17].”
LIMITED MEMORY BFGS METHOD FOR NONLINEAR MONOTONE EQUATIONS (2007, methodology): “Among numerous algorithms for solving systems of smooth equations, the Newton method, quasi-Newton methods, Levenberg-Marquardt method and their variants are particularly useful because of their fast local convergence property [5, 6, 7, 19].”
Historical development of the BFGS secant method and its characterization properties (2009, methodology): “We end this section by highlighting two additional results given by Dennis and Moré [35].”
Forward–backward quasi-Newton methods for nonsmooth optimization problems (2016, methodology): “The asymptotic analysis is based on an analogous of the Dennis andMoré theorem [26] for the proposed algorithmic scheme, and theBFGSquasi-Newtonmethod is shown to fit this framework.”
Let's Make Block Coordinate Descent Go Fast: Faster Greedy Rules, Message-Passing, Active-Set Complexity, and Superlinear Convergence (2017, background): “Indeed, considering the extreme case where we have one block containing all the coordinates, Newton updates can lead to superlinear convergence [Dennis and Moré, 1974] while matrix updates destroy this property.”
Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results (2011, methodology): “We also show that the well-known Dennis-Moré condition [7] on the Hessian approximation Bk is sufficient, and certain quasi-Newton formulae are thus appropriate.”