Convergence of quasi-Newton matrices generated by the symmetric rank one update

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

DeltaGrad: Rapid retraining of machine learning models (2020, background): “There exists a constant c 0 such that for all w 1 and w 2 , An assumption specifc to Quasi-Newton methods is the strong independence of the weight updates: the smallest singular value of the normalized weight updates is bounded away from zero (Ortega & Rheinboldt, 1970; Conn et al., 1991).”
Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results (2011, unknown): “It is also satisfied when using the symmetric rank one approximation to update Bk and the steps are linearly independent [1,3].”
Optimization Transfer Using Surrogate Objective Functions (2000, methodology): “…furnished by Davidon’s (1959) formula with constant c n and vector v n speciﬁed by Although several alternative updates have been proposed since 1959, the Davidon update (13) has recently enjoyed a revival among numerical analysts (Conn, Gould, and Toint, 1991; Khalfan, Byrd, and Schnabel, 1993).”
Quasi-Newton Methods for Deep Learning: Forget the Past, Just Sample (2019, methodology): “Brief Literature Review Quasi-Newton methods, such as BFGS [7, 20, 22, 52] and SR1 [11, 13, 29] and their limited-memory variants LBFGS [36, 44] and LSR1 [8, 38], respectively, have been studied extensively in the deterministic nonlinear optimization literature.”