Conditioning of Quasi-Newton Methods for Function Minimization

By David F. Shanno. 1970

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

Citations in the corpus

Fast large-scale optimization by unifying stochastic gradient and quasi-Newton methods (2013, methodology): “The ﬁrst is to use a quasi-Newton method (Dennis Jr & Mor´e, 1977), of which BFGS (Broy-den, 1970; Fletcher, 1970; Goldfarb, 1970; Shanno, 1970) or LBFGS (Liu & Nocedal, 1989) are the most common choices.”
Optimization Methods for Large-Scale Machine Learning (2016, methodology): “ochastic setting arising in machine learning. Before we embark on this discussion, let us review the basic principles underlying quasi-Newton methods, focusing on the most popular scheme, namely BFGS [27,60,68,146]. In the spirit of Newton’s method (6.4), the BFGS iteration for minimizing a twice co”
Entropy Search for Information-Efficient Global Optimization (2011, methodology): “…optimization algorithms: Trust-Region-Reﬂective (Coleman and Li, 1996, 1994), Active-Set (Powell, 1978b,a), interior point (Byrd et al., 1999, 2000; Waltz et al., 2006), and a na¨ıvely projected version of the BFGS algorithm (Broyden et al., 1965; Fletcher, 1970; Goldfarb, 1970; Shanno, 1970).”
The Visual Object Tracking VOT2016 Challenge Results (2016, methodology): “In case a solution satisfying the constraints was not found, a relaxed unconstrained BFGS Quasi-Newton method [32] was applied.”
COCO: a platform for comparing continuous optimizers in a black-box setting (2016, methodology): “• The BFGS algorithm [18, 29, 32, 65], as implemented in the Python function”
Pathfinder: Parallel quasi-Newton variational inference (2021, unknown): “The BFGS optimization algorithm is a quasi-Newton method that approximates the inverse Hessian through updates of positions and gradients of the objective function at positions along the optimization path (Broyden, 1970; Fletcher, 1987; Goldfarb, 1970; Shanno, 1970).”
Quasi-Newton Methods: A New Direction (2012, methodology): “And, if we exchange in the entire preceding derivation s ] y, B ] B−1, Bi−1 ] B−1 i−1, then we arrive at the BFGS method (Broyden, 1969; Fletcher, 1970; Goldfarb, 1970; Shanno, 1970), which ranks among the most widely used algorithms in machine learning overall.”
Accelerated Stochastic Matrix Inversion: General Theory and Speeding up BFGS Rules for Faster Second-Order Optimization (2018, methodology): “Quasi-Newton methods, in particular BFGS [4, 10, 11, 30], have been the leading optimization algorithm in various fields since the late 60’s until the rise of big data, which brought a need for simpler first order algorithms.”
Quasi-Newton Methods, Motivation and Theory (1974, unknown): “The most important instance of (7.24) was given by Broyden (1969), (1970), and independently by Fletcher (1970), Goldfarb (1970) and Shanno (1970).”
Quasi-Newton Methods for Deep Learning: Forget the Past, Just Sample (2019, methodology): “Brief Literature Review Quasi-Newton methods, such as BFGS [7, 20, 22, 52] and SR1 [11, 13, 29] and their limited-memory variants LBFGS [36, 44] and LSR1 [8, 38], respectively, have been studied extensively in the deterministic nonlinear optimization literature.”