Constrained Optimization and Lagrange Multiplier Methods

By Dimitri P. Bertsekas. 1982

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

Citations in the corpus

Frank-Wolfe Splitting via Augmented Lagrangian Method (2018, background): “It is possible to reformulate (OPT) into the problem of finding a saddle point of an augmented Lagrangian (Bertsekas, 1996), in order to split the constraints in a way in which the linear oracle is computed over a product space.”
The Numerics of GANs (2017, neutral): “See [4], Proposition 4.4.1.”
Understanding and mitigating gradient pathologies in physics-informed neural networks (2020, background): “More specifically, we consider minimizing the following loss in which the weights λ resemble the role of penalty coefficients in constrained optimization [41] L(θ) := Lr(θ) + M ∑”
Differentiable Model Compression via Pseudo Quantization Noise (2021, methodology): “Early network quantization methods focused on low-bitwidth networks such as BinaryNet Courbariaux et al. (2015; 2016), XOR-Nets Rastegari et al. (2016), or Ternary networks Li et al. (2016); Wu et al. (2018). Although these methods produce highly quantized models, their performance is not on par wit”
Taming VAEs (2018, methodology): “…of Lagrange multipliers, where we introduce the Lagrange multipliers λ ∈ R L and optimize the Lagrangian L λ via a min-max optimization scheme [45], A general algorithm for ﬁnding the extreme points of the ELBO F and of the Lagrangian L λ is provided in Proposition 1, where we extend the…”
Multi-population GWA mapping via multi-task regularized regression (2010, methodology): “The dual can now be expressed in the augmented lagrangian form (Bertsekas, 1982), to develop a much faster optimization technique than conventional gradient-based techniques.”
A binary level set model and some applications to Mumford-Shah image segmentation (2006, background): “See [18], [19] for details concerning the general algorithm.”
Physics-informed neural networks with hard constraints for inverse design (2021, methodology): “…k gradient In the -th iteration, we choose λ ki,j to generate exactly the that was previously generated in the ( k − 1)-th iteration by the penalty term [4, 37, 56], i.e., we require that and thus we have Similarly, for λ kh , we have [56] The initial values of all the multipliers are chosen as 0.”
Hyperspectral Image Restoration Via Total Variation Regularized Low-Rank Tensor Decomposition (2017, methodology): “This strategy of determining the variable μ has been widely used in the ALM-based methods [45], [46], which can facilitate the convergence of the algorithm.”
Geometric Applications of the Split Bregman Method: Segmentation and Surface Reconstruction (2010, methodology): “The use of inexact solvers in the context of differentiable problems is discussed in detail in [6], where conditions are presented under which multiplier methods can be guaranteed to converge.”
A complementary energy formulation of no tension masonry-like solids (2000, background): “In fact, the following equivalences hold hr6 0 is equivalent to hr max hr; ÿ k=a 0; 48 hrP 0 is equivalent to hrÿ min hr; ÿ k=a ÿmax ÿ hr; k=a 0; 49 where provided hr is dierentiable in its admissible subspace, the functions hr and hrÿ are dierentiable ”