The Convergence of a Class of Double-rank Minimization Algorithms 1. General Considerations

By C. G. Broyden. 1970

Subfields: Mathematical Optimization, Numerical Methods & Scientific Computing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Mathematical Optimization

In Numerical Methods & Scientific Computing

Citations in the corpus

Optimization Methods for Large-Scale Machine Learning (2016, methodology): “ochastic setting arising in machine learning. Before we embark on this discussion, let us review the basic principles underlying quasi-Newton methods, focusing on the most popular scheme, namely BFGS [27,60,68,146]. In the spirit of Newton’s method (6.4), the BFGS iteration for minimizing a twice co”
An empirical analysis of the optimization of deep network loss surfaces (2016, background): “The speed of gradient-descent-based algorithms are generally measured on strictly convex, in particular, quadratic functions [5, 10, 22, 23, 25].”
TOPAS and TOPAS-Academic: an optimization program integrating computer algebra and crystallographic objects written in C++ (2018, methodology): “For cases where calculating the elements of the A matrix is too computationally expensive then the BFGS method (Broyden, 1970; Fletcher, 1970; Goldfarb, 1970; Shanno, 1970) can be used to approximate the A matrix.”
Numerical Methods for Electronic Structure Calculations of Materials (2010, background): “One of the most successful updates from this class has been the BFGS [23, 56, 70, 183] update, which was discovered independently by Broyden, Fletcher, Goldfarb, and Shanno in 1970.”
A SURVEY OF NONLINEAR CONJUGATE GRADIENT METHODS (2005, methodology): “In the same way that quasi-Newton methods have been combined together by introducing parameters, as in the Broyden [ 7 ] family, CG methods can be combined together.”
Pathfinder: Parallel quasi-Newton variational inference (2021, unknown): “The BFGS optimization algorithm is a quasi-Newton method that approximates the inverse Hessian through updates of positions and gradients of the objective function at positions along the optimization path (Broyden, 1970; Fletcher, 1987; Goldfarb, 1970; Shanno, 1970).”
Practical tradeoffs between memory, compute, and performance in learned optimizers (2022, methodology): “For example, Newtons method has led to a number of approximations – e.g. diagonal approximations (Duchi et al., 2011), block diagonal (Martens and Grosse, 2015; Gupta et al., 2018), as well as a large family of quasi newton methods (Dennis and Moré, 1977) (e.g. BFGS (Broyden, 1970) and its low memor”