A Limited Memory Algorithm for Bound Constrained Optimization

By Richard H. Byrd, P. Lu, Jorge Nocedal, C. Zhu. 1995

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

Citations in the corpus

Unified Scaling Laws for Routed Language Models (2022, methodology): “We find the coefficients (a, b, c, d, Estart, Emax) as the best of repeated solutions provided by the L-BFGS-B algorithm [Byrd et al., 1995].”
Generative Visual Manipulation on the Natural Image Manifold (2016, methodology): “Projection via optimization: As both the feature extractor C and the generative model G are differentiable, we can directly optimize the above objective using L-BFGS-B [29].”
PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples (2017, methodology): “In fact, one of the advanced gradient-based optimization method—L-BFGS-B (Byrd et al., 1995)—usually produces purified images with even lower log-likelihoods than those of adversarial images (see Appendix E).”
Recommendations as Treatments: Debiasing Learning and Evaluation (2016, methodology): “For the experiments reported in this paper, we use Limited-memory BFGS (Byrd et al., 1995).”
Forecasting at Scale (2018, neutral): “(3) An important set of parameters in our model is C ( t ), or the expected capacities of the system at any point in time.”
DiffDock: Diffusion Steps, Twists, and Turns for Molecular Docking (2022, neutral): “This is also visible in the randomly chosen examples of Figure 9 and can be examined in our repository, where we provide all predictions of D IFF D OCK for the test set.”
DeltaGrad: Rapid retraining of machine learning models (2020, methodology): “We leverage the L-BFGS algorithm to approximate this, see e.g. Matthies & Strang (1979); Nocedal (1980); Byrd et al. (1994; 1995); Zhu et al. (1997); No-cedal & Wright (2006); Mokhtari & Ribeiro (2015) and references therein.”
Convolutional Kernel Networks (2014, methodology): “Since our work is not focused on speed, we run the L-BFGS-B algorithm for 4 000 iterations, which seems to ensure convergence of the objective function to a stationary point.”
Distributed optimization with arbitrary local solvers (2015, methodology): “We choose various values for H on two local solvers, CD [49, 56] and L-BFGS [10], which gave the best performance in general.”
DAGs with NO TEARS: Continuous Optimization for Structure Learning (2018, neutral): “As we describe in Appendix A, the precomputation time is only O ( m 2 p + m 3 ) where m p is the memory size of L-BFGS, and each coordinate update is O ( m ).”
Stochastic, Distributed and Federated Optimization for Machine Learning (2017, methodology): “These algorithms in general try to mimic the limited memory BFGS method (L-BFGS) [100], but model the local curvature information using inexact gradients — coming from the SGD procedure.”
Realtime performance-based facial animation (2011, methodology): “We use a gradient projection algorithm based on the limited memory BFGS solver [Lu et al. 1994] in order to enforce that the blendshape weights are between 0 and 1.”
Cross-Domain Object Recognition Via Input-Output Kernel Analysis (2013, methodology): “By adopting box constraints, we apply efficient off-the-shelf optimization approaches such as L-BFGS-B [19] to compute the DA-IOKL solution.”
Conceptual design of aeroelastic structures by topology optimization (2004, methodology): “Conn et al. (1991) is employed for updating the Lagrange multipliers, and the local minimization problems are solved by the quasi-Newton limited-memory algorithm of Byrd et al. (1995) .”
An Expectation Maximization Algorithm for Continuous Markov Decision Processes with Arbitrary Reward (2009, methodology): “For this work we will use a quasi-Newton optimization approach, the LBFGS-B algorithm [4].”
Root traits are related to plant water-use among rangeland Mediterranean species (2017, methodology): “We used a limited memory algorithm for bound constrained optimisation (Byrd et al. 1995) to estimate the two parameters which were not fixed in the models (mid-season length and”