A New Approach to Variable Metric Algorithms

By R. Fletcher. 1970

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

Citations in the corpus

Fast large-scale optimization by unifying stochastic gradient and quasi-Newton methods (2013, methodology): “The ﬁrst is to use a quasi-Newton method (Dennis Jr & Mor´e, 1977), of which BFGS (Broy-den, 1970; Fletcher, 1970; Goldfarb, 1970; Shanno, 1970) or LBFGS (Liu & Nocedal, 1989) are the most common choices.”
Optimization Methods for Large-Scale Machine Learning (2016, methodology): “ochastic setting arising in machine learning. Before we embark on this discussion, let us review the basic principles underlying quasi-Newton methods, focusing on the most popular scheme, namely BFGS [27,60,68,146]. In the spirit of Newton’s method (6.4), the BFGS iteration for minimizing a twice co”
Entropy Search for Information-Efficient Global Optimization (2011, methodology): “…optimization algorithms: Trust-Region-Reﬂective (Coleman and Li, 1996, 1994), Active-Set (Powell, 1978b,a), interior point (Byrd et al., 1999, 2000; Waltz et al., 2006), and a na¨ıvely projected version of the BFGS algorithm (Broyden et al., 1965; Fletcher, 1970; Goldfarb, 1970; Shanno, 1970).”
Online Batch Selection for Faster Training of Neural Networks (2015, background): “…and below all operations are element-wise 8: here, β 1 is taken to the power of t 10: here, β 2 is taken to the power of t 11: x t ← x t − 1 − α ˆ m t / ( √ ˆ v t + (cid:15) ) 12: until stopping criterion is met 13: return optimized parameters x t as the BFGS family of algorithms (Fletcher, 1970).”
Second Order Stochastic Optimization in Linear Time (2016, methodology): “Cheaper alternatives such as Quasi-Newton methods [Bro70, Fle70, Gol70, Sha70, Nes04] have been proposed and proven to be eﬀective.”
Quasi-Newton Methods: A New Direction (2012, methodology): “And, if we exchange in the entire preceding derivation s ] y, B ] B−1, Bi−1 ] B−1 i−1, then we arrive at the BFGS method (Broyden, 1969; Fletcher, 1970; Goldfarb, 1970; Shanno, 1970), which ranks among the most widely used algorithms in machine learning overall.”
Polynomial-Chaos-based Kriging (2015, unknown): “Local methods are usually gradient based algorithms such as the quasi-Newton Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm Goldfarb (1970); Fletcher (1970); Shanno (1970) and its modiﬁcations Byrd et al. (1999).”
A Survey of Optimization Methods From a Machine Learning Perspective (2019, methodology): “stu⊤ u⊤ ts ) + utu⊤ u⊤ t s . (52) The update formula of Ht+1 is HDFP t+1 = Ht − Htutu⊤tHt u⊤ t Htut + sts⊤ t u⊤ t st . (53) BFGS Broyden, Fletcher, Goldfarb and Shanno proposed the BFGS method [123], [124], [125], [126], in which Bt+1 is updated according to B(BFGS) t+1 = Bt − B ts ts⊤Bt s⊤ t Btst +”
COCO: a platform for comparing continuous optimizers in a black-box setting (2016, methodology): “• The BFGS algorithm [18, 29, 32, 65], as implemented in the Python function”
A UNIFORM ASTEROSEISMIC ANALYSIS OF 22 SOLAR-TYPE STARS OBSERVED BY KEPLER (2012, methodology): “The fit was iterated using a BFGS optimization algorithm (Broyden 1970; Fletcher 1970; Goldfarb 1970; Shanno 1970), which is a widely-used quasi-Newton “hill-climbing” nonlinear optimization method.”