Testing a class of methods for solving minimization problems with simple bounds on the variables

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

DeltaGrad: Rapid retraining of machine learning models (2020, background): “This has sometimes been motivated empirically, as the iterates of certain quasi-newton iterations empirically satisfy it (Conn et al., 1988). where s j 1 ,j m = max ( k Δ w j 1 k , k Δ w j 2 k , . . . , k Δ w j m k ) , has its minimum singular value σ min bounded away from zero.”
Finding Optimal Algorithmic Parameters Using Derivative-Free Optimization (2006, unknown): “are often recommended in the literature [8, 9, 11, 31, 33, 34].”
Convergence of quasi-Newton matrices generated by the symmetric rank one update (1991, methodology): “In two recent papers [1, 2], the authors describe and analyse a class of trust region methods for unconstrained and bound constrained minimization.”
The State of the Art (2001, unknown): “[22] (code LBFGSB), Conn, Gould und Toint [26] (code SBMIN gone into LANCELOT), [27], Moré and Lin [82], code TRON, Moré and Toraldo [88, 89] (code GPCG by Felkel, see [87]) and Felkels code PL2, again see [87] and Felkels PhD dissertation [39], see also Moré’s MINPACK 2 project [3].”
A Limited Memory Algorithm for Bound Constrained Optimization (1995, methodology): “We follow Conn, Gould and Toint [8] and stop the conjugate gradient iteration when the residual r̂ of (5.”