Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization

By C. Zhu, Richard H. Byrd, P. Lu, Jorge Nocedal. 1997

Subfields: Mathematical Optimization, Numerical Methods & Scientific Computing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Mathematical Optimization

In Numerical Methods & Scientific Computing

Citations in the corpus

Optimization Methods for Large-Scale Machine Learning (2016, background): “, those in [156, 93], since such general techniques may be less effective by not exploiting the structure of the reformulation (8.”
PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples (2017, methodology): “We found that one advanced methods in gradient-based constrained optimization, L-BFGS-B (Byrd et al., 1995) (we use the scipy implementation based on Zhu et al. (1997)), actually decreases p CNN ( X ) for most random initializations within the (cid:15) defend -ball.”
Physics informed deep learning for computational elastodynamics without labeled data (2020, methodology): “The whole training process consists of three stages, namely, 2,000 iterations of the Adam optimizer with 10 − 3 learning rate (LR), 2,000 Adam iterations with LR = 5 × 10 − 4 , and the L-BFGS-B optimizer until the loss converges to a small tolerance.”
DeltaGrad: Rapid retraining of machine learning models (2020, methodology): “Our approach is similar, but the technical details are very different, and more closely related to Quasi-Newton methods such as L-BFGS (Zhu et al., 1997).”
Incremental Few-Shot Learning with Attention Attractor Networks (2018, methodology): “We use an L-BFGS (Zhu et al., 1997) to solve the inner loop of our models to make sure W b converges.”
Visualizing Deep Convolutional Neural Networks Using Natural Pre-images (2015, methodology): “[12, 13] have used the L-BFGSB algorithm [54] to optimize their texture/style loss (8).”
Automatic differentiation in machine learning: a survey (2015, methodology): “When introducing new models, machine learning researchers have spent considerable effort on the manual derivation of analytical derivatives to subsequently plug these into standard optimization procedures such as L-BFGS (Zhu et al., 1997) or stochastic gradient descent (Bottou, 1998).”
Scalable Variational Gaussian Process Classification (2014, methodology): “For all the methods we used the L-BFGS-B optimizer [Zhu et al., 1997].”
Biological Sequence Modeling with Convolutional Kernel Networks (2017, positive): “We use the Adam algo-rithm [Kingma and Ba, 2015] to update the ﬁlters and the L-BFGS algorithm Zhu et al. [1997] to optimize the prediction layer.”
Few-shot learning of neural networks from scratch by pseudo example optimization (2018, methodology): “We used GPflow [22] with variational Gaussian approximation [24] and L-BFGS-B optimization [41] for GP inference, and utilized mean predictions as pseudo supervisions.”
Texture synthesis and the controlled generation of natural stimuli using convolutional neural networks (2015, methodology): “In our work we use L-BFGS [22], which seemed a reasonable choice for the high-dimensional optimisation problem at hand.”
Pathfinder: Parallel quasi-Newton variational inference (2021, unknown): “There are several prominent implementations of L-BFGS that vary in their details; we use the version introduced by Byrd et al. (1995) and detailed by Zhu et al. (1997).c The pseudocode for Zhu et al.’s version of L-BFGS (without bounds) is listed in Algorithm 6 in Appendix F.”
Worst case optimization: a method to account for uncertainties in the optimization of intensity modulated proton therapy (2008, methodology): “The optimization is done using the L-BFGS-B code (Zhu et al 1997).”
The statistical finite element method (statFEM) for coherent synthesis of observation data and model predictions (2019, methodology): “red to as the maximum posteriori (MAP) estimate is given by w = argmax w p(Yjw)p(w): (49) This, often non-convex, optimisation problem can be numerically solved with conventional algorithms, see e.g. [47]. Alternatively, as in this paper, we can use the expectation of the parameter vector w = E[w] =”
Texture Synthesis Using Convolutional Neural Networks (2015, methodology): “In our work we use L-BFGS [30], which seemed a reasonable choice for the high-dimensional optimisation problem at hand.”
Remark on “algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound constrained optimization” (2011, methodology): “1 Subspace Minimization Phase As indicated in Section 4 of [6], the algorithm performs the subspace minimization phase by minimizing a (primal) model of the objective function.”