Numerical methods for unconstrained optimization and nonlinear equations

By John E. Dennis Jr., Bobby Schnabel. 1983

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

Citations in the corpus

Gradient Descent Happens in a Tiny Subspace (2018, methodology): “Similarly, given the nontrivial relationship between the Hessian and the gradient, a natural question is whether there are any practical applications for second-order optimization methods (see Bottou et al. (2016) or Dennis Jr & Schnabel (1996) for a review).”
Optimization Methods for Large-Scale Machine Learning (2016, methodology): “roach the minimizer, but from any initial point w 1 the iteration (6.2) with B = (r2F(w 1)) 1=2 and 1 = 1 will yield w 2 = w . These latter choices correspond to a single iteration of Newton’s method [52]. In general, it is natural to seek transformations that perform well in theory and in practice.”
Automatic differentiation in machine learning: a survey (2015, methodology): “Compared with the straightforward simplicity of forward accumulation mode, reverse mode AD can, at first, appear somewhat “mysterious” (Dennis and Schnabel, 1996).”
Interior Methods for Nonlinear Optimization (2002, background): “7 and related results may be found in, for example, [25, 28, 34, 52, 77, 80].”
Stochastic Gradient Descent Tricks (2012, background): “Under sufficient regularity assumptions, when the initial estimate w0 is close enough to the optimum, and when the learning rate γ is sufficiently small, this algorithm achieves linear convergence [6], that is, − log ρ ∼ t, where ρ represents the residual error.”
Inverse and Optimization Problems in Heat Transfer (2006, methodology): “However, for some realistic cases, the variation of the objective function with the search step size is not unimodal and, then, more robust techniques are required, such as the exhaustive search method or a technique based on exhaustive interpolation [24-27,39,40].”
Simulation of Unilateral Constrained Systems with Many Bodies (2005, methodology): “Dennis and Schnabel [ 22 ] propose a more elaborate approach using a QRinstead of a LU-decomposition.”
The effect of cavitation on glacier sliding (2005, methodology): “A solution of the sliding problem can then be reduced to solving the constraints (4.9), (4.10) and (4.14) numerically, which is achieved using a backtracking line search modification of Newton’s method (Dennis & Schnabel 1996, ch. 6).”
Recurrent motions within plane Couette turbulence (2006, methodology): “The radius of the trust region is varied from step to step by comparing the actual reduction in error following the step with the reduction in error predicted by the linearization (Dennis & Schnabel 1996).”