Numerical Optimization (Springer Series in Operations Research and Financial Engineering)

By Jorge Nocedal, Stephen J. Wright. 2000

2,414 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

Citations in the corpus

From Pixels to Torques: Policy Learning with Deep Dynamical Models (2015, methodology): “The cost functions are then minimized by the BFGS algorithm (Nocedal & Wright, 2006).”
Graph Kernels (2008, neutral): “Furthermore, if computing matrixvector products is cheap — because M is sparse, for instance — the CG solver can be sped up significantly (Nocedal and Wright, 1999).”
Learning deep dynamical models from image pixels (2014, methodology): “The cost functions are then minimized by the BFGS algorithm [11].”
Distributed optimization of deeply nested systems (2012, methodology): “The COIL–20 image dataset (Nene et al., 1996), commonly used as a benchmark to test dimensionality reduction algorithms, contains rotation sequences of 20 different objects every 5 degrees (i.e., 72 images per object), each a grayscale image with pixel intensity in [0, 1].”
Stochastic, Distributed and Federated Optimization for Machine Learning (2017, methodology): “CD Coordinate Descent [148] APPROX Accelerated, Parallel and Proximal Coordinate Descent [59] GD Gradient Descent with Backtracking Line Search [134] CG Conjugate Gradient Method [77] L-BFGS Quasi-Newton with Limited-Memory BFGS Updating [27] BB Barzilai-Borwein Gradient Method [9] FISTA Fast Iterat”
Variable Metric Stochastic Approximation Theory (2009, methodology): “The BFGS algorithm (Nocedal and Wright, 1999) was developed independently by Broyden, Fletcher, Goldfarb, and Shanno in 1970.”
Scaling Up Quasi-newton Algorithms: Communication Efficient Distributed SR1 (2019, methodology): “Contrary to arguably most popular quasi-Newton method, (L)BFGS [37, 42, 43], the (L)SR1 method does not enforce that the Hessian approximations are positive definite, and as such is usually implemented with a trust-region [43].”
Second-Order Information in Data Assimilation* (2002, background): “Clearly, if D is defined as the solution of ( ∂F ∂Z )T · D = ∂G ∂Z (71)”
Meta-Learning with Implicit Gradients (2019, methodology): “This CG based inversion has been successfully deployed in Hessian-free or Newton-CG methods for deep learning [36, 44] and trust region methods in reinforcement learning [52, 47].”
Accelerated Parallel Optimization Methods for Large Scale Machine Learning (2014, methodology): “Much effort has been put into developing optimization methods for this model, ranging from coordinate minimization [5], randomized coordinate descent [18], stochastic gradient descent [2, 17], dual coordinate ascent [19], to higher order methods such as interior point methods [6], L-BFGS [15], to na”
Distributed average consensus with least-mean-square deviation (2007, neutral): “We start with the identity which can be verified by multiplying out, and noting that J 2 = J and J W = W J = J .”